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residual disadvantages were between --7.7 and -11.0 percent for the 
four age groups. Blacks performed best on those science exercises 
most dependent upon daily experience and common knowledge, and 
poorest on those which involve a detached research attitude toward 
the objects and phenomena of science. - (Author/ JR) 
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SUMMARY 



The main purpose of National Assessment is to measure what 
J nation's children and young adults know and can do, in order 

see what changes occur from time to time. Science was 
sessed for a national sample of respondents in 19G9--70. After 
ience has been assessed agaiii in 1972-73, reports of science 
suits will emphasize what seems to have been learned better, 
d what less well. In the meantime, our reports necessarily are 
confined to the results of the only administration of science 
exercises conducted so far. ^ 

National Assessment Report 1 gave national perce^ftages of 
success for science exercises. Report 4 examined percentages of 
success for groups, comparing percentages correct between the 
two sexes, among four geographical regions and among four sizes 
of community. 

The present report extends the results to cover the perfor- 
mance of Blacks, and of respondents with differing levels of 
parental education and from differing types of community. For 
each group on each of these characteristics, we describe perfor- 
mance differences as they are . In addition, for each group, 
including those examined m Report 4, we discuss differences as 
we estimate they might have been if the other four character is-- 
tics had been proportionately represented in each group. (See 
appendix B for further discussion) . 

Selection of Summary Values 

Science exercises were administered at the student ages — 
9f 13, and 17 — and to young adults between the ages of 26 and 
35. Roughly 100 exercises were administered at each age. For 
each exercise at every age, we focus upon the percentage of 
correct responses. 

To represent the typical performance level for a single 
exercise for a given group — e.g., females, Northeasterners , 
residents of small cities — it is natural to consider the dif- 
ference between % success (percentage; of correct responses) 
for that group and % success for all respondents of that age. 
As a summary for all exercises, the median difference — the 
midpoint of the differences for all exercises — is appropriate 
and effective. 

The summary values reported in the section were chosen so 
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as to emphasize general behavior and minimize group disparities 
that mi'jht be misleading. To represent typical performance of 
a given group, in this summary we report jusc olie., number for 
students and another number for young adults. Eiich of the 
ages 9, 13, and 17 has a median difference; we choose the middle 
one of these as the summary number for students. The summary 
value reported for adults is the median difference for young 
adults. While these summaries tell the general story, examina- 
tion of individual exercises and summarization of the differences 
in behavior between groups of exercises is also ne^eded . 

Limitations of Unadjusted Results 

In the previous report of group results on Science (National 
Assessment Report 4) the following cautionary statement was made: 

There is a kind of interpretation that should never 
be made on the basis of the sort of figures given in 
this report. The fact that figures reflect Southeast 
performance or Big City performance does not mean 
that the performances thus reflected have arisen 
precisely from living in the Southeast or in a Big 
City, or from the attitudes, techniques, facilities 
and staffs of the school system involved. In 
particular, just what happens in a region involves 
other things than that region'?: schools. Larger 
fractions of the children in some regions belong to a 
particular size of community group. Thus effects 
due only to size of coTimunity can appear to be regional 
differences. Larger or smaller fractions of the 
parents in some regions have particular amounts Qf. 
education. Thus effects due only to parental education 
can appear to be regional differences . And so on 
Migration from one region or size of community to 
another can further complicate the picture. There 
are such difficulties, some of which we know "how to 
adjust for, and some of which we do not. 

Great caution should also be exercised in interpreting the 
results for the group varir.bles from the present report. 
Attempts, based on results reported here, to assign causation 
to any particular set of educational factors,, or to any variables 
from the heredity-en vironjnent complex would be unjustified. 

Unadjusted Results 

The unadjusted results are shown graphically in exhibit 
S-1 and numerically in exhibit S-2. 
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Summary Values of Unadjusted Results for Groups 
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*The three extreme size and type of community groups labelled 
in capital letters were specially selected to each include about 
10% of all respondents and to be as extreme as could be con- 
veniently defired. Definitions for all the groups appearing in 
this exhibit may be found in Appendix D, 
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Summary Values of Unadjusted Results for Groups 







Student 


Adult 


Group 


Median Effect 


Median Effect 


Females 




-1.4% 


( -4.6%) 


Males 




1.5% 


( 5.8%) 


Southeast 




-4.9% 


( -4.9%) 


West 




0.0% 


( 2.4%) 


Central 




1.6% 


( .7%) 


Northeast 




2.3% 


( .6%) 


INNER CITY* 




-13 . 7% 


(-10. 2%) 


RURAL 




-6.1% 


( -4.7%) 


Rest of Big City 




-2.5% 


{ -2.9%) 


Small City 




.9% 


( -2.7%) 


Medium City 




1.3% 


( 0.4%) 


Suburban Fringe 




2.7% 


( 0.8%) 


AFFLUENT SUBURB 




6.3% 


( 10.9%) 


Parents; no high school 




-8.4% 


( -7.9%) 


Parents; some high school 




-6.1% 


( -1.6%) 


Parents; graduated high school 




0.1% 


( 3.0%) 


Parents; beyond high sciiool 




5.2% 


( 9.1%) 


Black 




-14.5% 


(-15.8%) 


Non-Black 




+ 3.1% 


( +2.7%) 


The largest differences — those between 


one extreme 


group of 


a kind and the other — are: 








Sex 




2.9% 


( 10.4%) 


Region 




7.2% 


( 7.3%) 


Size and type of community 




20.0% 


( 21.1%) 


Parents' education 




13.6% 


{ 17„0%) 


Color 




17.6% 


( 18.5%) 


* The three extreme size and type 


of community groups labelled 


in capital letters were specially 


selected to each 


include 


about 10% of all respondents and 


to be 


as extreme 


as could be 



conveniently defined. Definitions for all the groups appearing 
in this exhibit may be found in appendix D. 
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"Double Counting" 



Blacks are more frequent in extreme inner cities, than in 
the country as a whole, persons v/hose parents had no high school 
are more frequent in extreme rural areas than in the country as 
a whole, per.-^ions whose parents went beyond high school are more 
frequent in extreme affluent suburbs, and so on. If persons 
from extreme affluent suburbs do well, we can expect that 
those from families with high parental education will do well 
also. Some of what the unadjusted results mean for both of 
these groups comes from high performance by exactly the same 
persons . 

When we want to think about more than one kind of grouping 
at the same time, such "double counting" can be both misleading 
and confusing. A simple way to ensure that individuals get 
counted only once is to assign part of any observed performance 
difference to each of the subgroups concerned. This is one way 
to look at the construction of balanced effects . 

When we do this, it is natural to m.ake the sum of the 
parts add up to the whole. The unadjusted effects: 



should not be added together, since an extra proportion of 
students are incj.uded in both. The balanced effects: 



can be added together much more sensibly. (The sum +7.3% = 
3.3% + 4.0% is the comparative performance of those students 
who both live in an extreme affluent suburb and have a parent 
educated beyond high school.) 

Limitations of Balanced Results 

Planning the improvement of our educational system would 
be very much easier if we knew how much change in percent 
successful performance would come about if we could and did 
change various factors, such as parents' education. The 
practical, dollar-on- the-barrel-head form of question would be 
exactly this: how much change in product could we buy by making 
each of certain specified changes in schools, communities, or 
homes? Neither the results of National Assessment or those of 
any other study of "what now is" of f ers trustworthy estimates 
of amount of change in educational performance for given changes 
in input. Only experiments, which at best would be difficult to 
carry out with sufficient care — and which at worst would be 
impossible — could provide such estimates. 



extreme affluent suburb 
parents beyond high school 



+ 6.3% 
+ 5.2% 



extreme affluent suburb 
parents beyond high school 



+3.3% 
+ 4 . 0% 



ERIC 



XV 



Every factor National Assessment has been able to use as a 
basis for analysis stands not only for the factor indicated by 
its name, but also for a variety of other factors National 
Assessmei^t did not, or could not measure--other factors which 
are associated, in today's population, with the named factor. 
Each effect, unadjusted or balanced, combines the influences 
of factors, some idantifiod and others not (a somewhat smaller 
group, of course, for each balanced effect than for the corres- 
ponding unadju£5ted effect) . 

Although the complete picture of all the factors deter- 
mining educational achievement is certainly not known, we have 
strong evidence that a wide variety of factors are involved 
and we have a good idea of what many of these factors are. Some 
examples: (1) approval of learning by family and peers; 

(2) innate intelligence; (3) adequate health and nutrition; 

(4) a stimulating learning environment in the school (and, 
possibly, in the neighborhood) ; and (5) an appreciation of 
education and its relevance by the student. 

These examples are all rather abstract and National Assess- 
ment did not measure such variables. It is true, however, that 
some of the factors collected are — or might reasonably be presumed 
to be — sensitive to such variables. Indeed the groups of stud/^nts 
differentiated by such National Assessment factors as region, 
STOC, and parents' education do differ markedly in their achievG- 
ment. However, some of the variables one might consider to be 
of primary importance are not tapped in any clear way by the 
factors National Assessment collected. Thus, even though it may 
be incomplete or inaccurate, the child's statement of his or her 
father's and mother's education probably measures to some extent 
parental approval of education, but none of our data are 
explicitly tied to the approval or disapproval of the child's 
peers. If this is an important determinant of achievement, its 
effects might easily masquerade as part of the effect of color 
or extreme inner city or both. 

:ven the variables that are in the analysis may not be ji..st 
what they seem. Thus, parental education may be a less good 
indico tion of parents ' positive contributions to their child ' s 
education than is the type and size of their chosen community. 

For such reasons, we cannot presume that the names we use 
to describe our factors describe what they measure, so far as 
our analyses go. In interpreting the balanced results we must: 
(1) remember the many important variables that were not measured, 
and, (2) realize that much of the effect our analysis seems to 
attribute to some factors may be due to other factors or other 
variables. So long as we recognize that balanced effects havc^ 
these sorts of difficulties, we can gain by looking at the 
balanced results — sometimes in place of the unadjusted ones and 
sometimes in comparison with them. 
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Even balanced effects, of the sort and quality provided 
by this study, are not sufficient to firmly establish — or 
closely measure the extent of — causation by any particular factor 
or factors. 

We must refrain from attempts to ascribe firmly established 
causation to any particular set of educational factors or to any 
variables from the heredity-environment complex. If we look at 
the balanced results, we avoid a few specific difficulties, of 
which "double counting" is an example. We leave untouched a 
variety of other difficulties whose size we do not know how to 
assess. Used tentatively, however, the balanced results are 
probably better guides to the operation of the mechanisms 
involved, while the unadjusted results show more clearly the 
problems that the people and the schools of this country face. 

For Science 1970-71, balanced effects are smaller than 
unadjusted ones. In fact, considering the limited number and 
character of the factors actually measured and analyzed , these 
restrictions are quite large. Some were reduced to less than 
half of their unadjusted values. Indeed, it is not unreasonable 
to speculate that if additional (and more directly relevant) 
factors had been available for the analysis, some differences 
might have been reduced to negligibility. For example, in the 
analysis reported here we use education of parent as one of the 
balancing variables when looking at type of community effects. 
The measure of parent education was a four category representa- 
tion of years of schooling. In terms of effects on a child's 
performance on science exercises, however, a much more relevant 
measure of the home influences on the child's learning might 
have been the kind and quality of the parents' education — or 
the attitude of the parents toward the value of an education-- 
or the time spent by the mother in reading to the child in the 
years before he entered school — or the number and kinds of 
books and magazines in the home. 

Of course, as additional factors that reflect the same 
kinds and qualities of influences are introduced into an analysis, 
the effect of any particular additional factor is likely to be 
smaller and the direction of its effect harder to predict. 

Thus, speculations concerning the effect of balancing for 
additional factors have to be only speculations. 

Besides the absence of important factors , we need to 
notice that the factors we have used (1) are measured only in 
coarse subgroupings and (2) are measured with varying amounts of 
error. The presence of these difficulties uj:ually leads to 
smaller reductions by balancing than would be observed in their 
absence. 

If we are fortunate, the change from unadjusted to balanced 
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Exhibit S-3 
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*The three extreme size and type of community groups labelled in 
capital letters were specially selected to each include about 10% 
of all respondents and to be as extreme as could be conveniently 
defined. Definitions for all the groups appearing in this exhibi 
may be found in Appendix D. 
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is a step in the right direction, but we do not know what 
fraction of the way it has taken us. 



Balanced Results 

The balanced results are shown graphically in exhibit S-3. 
and numerically in exhibit S-4. 



Exhibit S-4 

Summary Values of Balanced Results for Groups 

Student Adult 
Group Median Effect Median Effect 



Females 


-1. 


1% 




-4.4%) 


Males 


1 . 


3% 




4.8%) 


South ea. st 


-2 


5% 






West 




5% 




1.9%) 


Central 




0% 




.3%) 


Northeast 




8% 
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.3%) 


INNER CITY 


-5. 


5% 




-3.2%) 


RURAL 


-2. 


8% 




-4.5%) 


Rest of Big City 


-1. 


5% 




-1.9%) 


Small City 




5% 




-2.0%) 


Medium City 




6% 




.8%) 


Suburban Fringe 
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AFFLUENT SUBURB 


3 
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5 • 6 % ) 


Parents: no high school 


-7 . 


3% 




-5.5%) 


Parents: £'ome high school 
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9% 




-1.0%) 


Parents: graduated high school 
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Parents: beyond high school 
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Black 
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10.9%) 


Non-Black 


2. 


1% 




2 . 0%) 


The largest differences-- those between 


one , 


extreme 


group 


of a 


kind and the opposite extreme--are : 










Sex 


2. 


4% 
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9.2%) 


Region 


3 . 


5% 


( 


4.3%) 


Size-and-type ol coinirtunity 


8 . 


8% 


( 


10.1%) 


Parents ' education 


li. 


3% 


( 


12.4%) 


Color 


12. 


3% 


( 


12.9%) 



* The three extreme size and type of community groups labelled 
in capital letters were specially selected to each include 
about 10% of all respondents and to be as extreme as could be 
conveniently defined. Definitions for all the groups appearing 
in this exhibit may be found in appendix D. 
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These are appreciably reduced from the corresponding differences 
among unadjusted effects. The extreme differences for sex and 
for parents' ediication are ab">ut 5/6 as large after balancing 
as they were before , that for color is about 2/3 , and the 
extreme balanced differences for region and for size and type 
of community are about 1/2 of their adjusted covin terparts . 

After balancing, only one region and only the ttiree extreme 
size and type of community categories show large effects. 
Indeed, if we consider only: (1) the three regions other than 
the southeast, and (2) the four sizes and types of comiriunity 
other than the three extremes, the largest differences are: 

balanced unadjusted 

3 regions .5% (1.6%) 2.3% (3.1%) 

4 sizes & types 2.3% (2.7%) 5.3% (3.7%) 



This leads to the following comparisons of largest differences: 



balanced 



unadjusted 



3 regions 

4 sizes & types 
sex 

Southeast (*) 
Extreme types 
Parents ' education 
Color 



0.5% 
2.3% 
2.4% 
-3.3% 
8.8% 
11.3% 
12.3% 



2. 

5. 

2. 
-6. 
20. 

13 , 

17 , 



3% 
3% 
9% 
5% 
0% 
5% 
b% 



{^) compared with median of other three regions. Clearly 
large differences appear only for: 



extreme types of community 

parents * education 

color 



no matter whether we look at balanced or unadjusted results. 



Differences Between Kinds of Exercises 



Respondents reporting neither parent completing high school 
had exceptional difficulty on exercises requiring special vocab- 
ulary knowledge or reasoning from graphs and pictures. 

Those exercises on which Blacks performed relatively better 
seemed to involve knowledge of common phenomena — particularly 
exercises whose answers might be gained from daily experience. 
Those exercises on which Blacks performed relatively poorer 
seemed to involve a detached research attitude toward the objects 
and phenomena of science--exercises whose cinswers are more 
lively to come from school experience . 
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As in National Assessment Report 4, there is a slight 
tendency for exercises assigned to bb^iective 2 {possess the 
abilities and skills needed to engage in the processes of 
science) to display stronger differences between groups than 
exercises assigned to objective 1 (know the fundamental facts 
and principles of science) . Among the variables newly analyzed 
in this report, this tendency is most clear for color results. 

Limitations of Response and Measurement 

Two other spiecial lim.itations should be noted that apparent! 
contribute to some irregularity in the patterns of results for 
age 17, The out-of -school sample included at this age suffers 
from the problem that many teenagers are difficult to locate at 
home. Among the subgroup of 17-year-olds who cannot readily be 
found or will not cooperate are probably a large number of 
individuals who have been quite unsuccessful in school and are 
antagonistic to scholastic tasks. Thus the 17-year-olds 
available in the sample might be expected to perform well com- 
pared to unavailable 17-year-olds. The' problem is compounded 
by the possibility that certain subgroups might be unavailable 
in greater proportion than other subgroups. This may be the 
case with 17-year-old Blacks, for example, since the relative 
number of Black males in the 17-year-old sample is only about 
3/4 the number at ages 9 and 13, Thus, the 17-year-old Black 
sample might be selectively different from the Black samples 
at other ages. In fact, the results to be examined in chapter 2 
show that Black 17-year-olds do perform relatively better than 
Blacks at other ages. 

Similar difficulties occur for the young adult sample, 
where 44% of those approached refused to participate, 

A further difficulty at age 17 is that there happened to 
have been a larger number of very difficult exercises than at 
other ages, (See page 10 of Report 1). To the extent that 
very difficult exercises differentiate subgroups less sharply 
than other exercises, the differences among subgroups will be 
reduced at age 17, 

Information about student's parents' education was obtained 
either from students or from school records , At ages 9 and 13 
many respondents could not be assigned to parents' education 
subgroups and some that were assigned were undoubtedly assigned 
in error. 
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CHAPTER 1 

THE MERITS AND WEAKNESSES OF ADJUSTMENT (.INCLUDING BALANCING) 



The educational administrator wants to make comparisons 
between groups, to find out who is learning more and who less, 
in hope of being able to improve performance in the lagging 
groups. Indeed, he would like to go further and find out what 
factors to change and how much changes in these factors: would 
strengthen the educational achievement of the students affected - 
For example, when we find that boys know less about the repro- 
ductive system of both sexes than do girls, this raises at once 
the question of strengthening the education of boys. Inevitably 
the desire is to subdivide the country into finer and finer 
groups so as to compare subgroups that have "everything alike" 
except the variable being studied. 

In other words, we search for causes of the differences. 
Unfortunately, we cannot have "everything alike" in social 
problems and rarely in physical problems either, and so we are 
not actually able to carry out the precise prog^-am. But half a 
loaf may be better than none, and so we may carry out that 
part of the program that seems feasible. We subdivide by 
important variables and make comparisons in performance among 
subgroups . 

One thing that happens is that as we introduce several 
variables the number of subdivisions grows like a product. For 
example, if we have 5 variables with 2, 3, 5, 1 and 4 categories 
respectively, we have 2x3x5x7x4 = 840 giroups , and a 
sample of 8400 people would give an average of only 10 per group. 
Naturally many groups would be empty and many fuller than 10, 
but it may now be hard to make comparisons among groups, for some 
will be too sparse. 

We might try to avoid these sparse cells by looking at 
factors one at a time. 

However , children in the extreme affluent suburb tend , more 
than children in the extreme inner city , to have better aducated 
parents. Because of this lack of balance, part of the difference 
between these two groups may be considered as growing out of the 
difference in parental education. 

It is natural to ask, "What would the difference between 
These extreme types of communi.ty have been if the distribution 



of parental education, sex, color, and region had been the same 
for both types of community referred to above?" Were it possible 
to rearrange the world to equate these distributions for each 
type of community, the effects upon our nation and its schools 
^would be profound. Such rearrangement is not possible. it is 
usually appropriate to think of the balanced results presented 
in this report as reflecting the differences we would see in the 
absence of masquerading by the other four factors. VJe can be 
reasonably sure that the balanced results do a much better job 
than the unadjusted results of reflecting such differences. 
Appendix B presents more details about the nature and computa- 
tional procedures of balancing. 

Still another questioii concerns the combination of factors . 
The performance of a given subgroup may be found to differ, 
depending upon subgroupings on other variables. Thus, the 
effect associated with extreme affluent suburbs may be different 
in the Northeast and the Southeast. Or the effect associated 
with sex may be somewhat different for Blacks and non-Blacks • 
These interactive differences are not considered in this report 
and balancing does not adjust for them. 

It is natural to ask whether this or any such method of 
analysis can help us. To some extent they can aid, to some 
extent not. >tfe cannot make up for cases we don't have but we 
may be able to supply approximate analyse.s that will come near 
to answering such a question as what is the effect of region of 
the country on performance when you control for STOC and several 
other variables. If the effect of region is substantially 
reduced by the analytical adjustment, we may be inclined to th^.nk 
that region is not in itself the cause of the raw differences as 
much as the other variables. One role of ad justment then is to 
help us make approximate comparisons and summaries that we 
cannot make by directly subdividing all the variables. In this 
report we exploit extensively one special method of adjustment 
called balancing^ which we explain in appendix B.. 

Elsewhere we have many cautionary remarks about the dangers 
of misinterpreting the causative powers of given background 
variables, for they may be poorly measured and they may not mean 
what they say. For an example fiom the field of warfare, in 
World War li the more fighter opposition that bombers had, the 
closer to the target were the bombs^ Why? Fighters didn't 
come up when the weather obscured the target. Such proxy vari- 
ables, especially when their implication may be the absolute 
reverse of their obvious logical effect/ puts us in grave danger 
of making mistakes. We do not go further*" into that here. 

Nothing but experimentation, if that, can serve to demon- 
strate what the actual effect of changes will be. We are, 
however, trying to get hints and insights from the data we have. 




Furthermore, if someone does have a causative model involving 
the variables National Assessment measures, he does have a chance 
to check it against these results. 

We see then that the purpose of analysis and adjustment is 
to help the data reveal information that they cannot give in 
their raw form. Aside from the dangers of misinterpretation, we 
have the political arguments for and against adjustmeTit. First, 
against: if adjustment for background variables seems to reduce 
the differences between a group of the population and the national 
average, it has been argued that this tends to minimize the dis- 
advantage of the group and j it is further argued , that adjustment 
should not be made. The direction of the effect of an adjustm.ent 
is not necessarily one-way; adjustments can increase differences 
as well as decrease them. Those arguing against adjustment in 
the reduction case would presumably argue for it in the case of 
increased discrepancies . 

A second argument favors adjustment . It argues that we 
must adjust for important variables (presuming that the adjust- 
ment will reduce effects) so that we show the potential of the 
disadvantaged group . 

Clearly the people making the first and second argument 
want the same thing, to improve the position of the disadvan- 
taged group, and of course, this is a national gonl. Steps 
toward achieving such goals do depend on searching for causes 
and methods of improvement, on finding weak spots in a system 
and so on. We should therefore look at our data in every way 
we can for hints about how the system works and how to improve 
it. Analysis and adjustment are tools for doing this. The 
question is not whether to adjust or not. but, "What are the 
useful ways?", "What do the variables mean?", "What further 
variables do we need to measure?", and "How shall we interpret 
the results?" 
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CHAPTER 2 



COMPARATIVE PERFORMANCE OF BLACKS 



In view of the widespread national concern with the educa- 
tional disadvantage of Blacks and other minority groups, National 
Assessment v;as designed , so as to begin an evaluation of the 
details of such possible disadvantage. The color of each 
respondent was noted by the exercise administrator on the 
assessment package as answers were turned in. During year 01, 
color was noted as "Black" or "White" or "Other" by the 
administrator. The non-Black category in addition to many 
Whites, included Orientals, Mexican-Americans, Indians and 
some Puerto Ricans. The present chapters give results for 
Blacks in this first cycle of Science assessment. The standard 
measure of relative performance on each exercise is the dif- 
ference between the percentage of correct answers given by 
Blacks and the national percentage of correct answers. These 
measures give benchmarks for evaluating potential progress in 
later assessment cycles. 

The performance of the Black respondents in National 
Assessment can reflect individual, school or environmental 
factors, and there is no sure way to disentangle the complex 
web that represents the Black educational condition in 
America today. In the next chapter, a preliminary attempt is 
made to adjust Black results to take account of the general 
advantage for Whites of disproportionate representation in 
affluent suburbs, higher numbers of highly educated parents and 
so on. 

The effects for Blacks and non-Blacks differ widely from 
exercise to exercise, as noted in National Assessment Report 1 
(pp. 151, 155) . In this and the next chapter we discuss 
individual exercises and types of exercises on which Black 
respondents perform atypically better or worse than their general 
level. 

Exhibit 2-1 shows the distribution of relative performances 
by Blacks over all exercises at the four age levels. The median 
results for Blacks are consistently belov: national results, 
by 14.5%, 15.0%, 11.8%, and 15.8% at ages 9, 13, 17, and young 
adults (26-35). For individual exercises, the spread around 
these medians is considerable, however, as the exhibit shows. 
The smaller median deficit at age 17 than at the other three 
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Exhibit 2-1 




ages is difficult to interpret. In any case, it seems safe to 
say that there is no systematic decline in relative Black per- 
formance with increasing age. (This conclusion also applies if 
attention is focused solely on exercises administered at more 
than one age level.) 



Median relative performances by Blacks on exercises for 
each science objective are shown in exhibit 2-2. Objectives 1 
and 2 covered different forms of science achievement with 
sizeable numbers of exercises. Objectives 3 and 4 touched upon 
attitudes and orientations toward science and scientists , 
employing rather few exercises. Comparing objective 1 (know 
the fundamental facts of science) with objective 2 (possess the 
abilities and skills to engage in the processes of science) we 
note that at all four ages the Black disadvantage is consistently 
less for objective 1 , * by amounts of 2.2%^ 3.7%, 8.7%, and 
3.0%. That is. Black respondents at all assessed age levels tend 
to perform relatively better on science fact exercises than on 
science process exercises. 

At ages 13 and 17, there is a tendency for the median Black 
deficit to be rather small (4 to 6%) for the exercises of 
objective 4 (show appreciation of scientists and science) . 
At age 17, the deficit is also small (4%) on objective 3 
(understand the investigative nature of science) . There are 
relatively few exercises under these two objectives, but the hint 
is nevertheless clear that whatever their achievement levels. 
Black teenagers do not necessarily express a lack of appreciation 
for science. 



Median results for physical science vs. biological science 
exercises are presented in exhibit 2-3. There are no systematic 
differences in Black performance on these two types of science 
exercises . 



^Region, sex, and size of community groups also are less strongly 
differentiated on objective 1 than objective 2 exercises. See 
Report 4 , pp . 10 , 5 , 21 . 



Exhibit 2-2 



Median Relative Performance by Blacks for 
Exercises Under Each Objective 

Science 

Objective 9 13 17 Adult 

1. Facts and -15.5% -15.0% -10.9% -14.8% 

Principles (96)* (75) (89) (85) 

2. Abilities and --17.7% -18.7% -19.6% -17.8% 

Skills (29) (31) (24) (24) 

3. Understand 

Investigative -12.1% -18.8% -4.2% -23.7% 

Nature (11) (8) (6) (5) 

4. Attitudes and -10.7% -4.3% -6.0% -10.6% 

Appreciations (9) (3) (5) (5) 

^Numbers of exercises in parentheses 

Note: Minus percent figures represent median percentages 
below national averages. 
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Exhibit 2-3 



Median Relative Performance by Blacks for 
Physical and Biological Science Exercises 



Physical 
science 

Biological 
science 



13 



-16 . 0% 
(78)* 

-14.2% 

(48) 



■15.0% 
(67) 

•15.4% 
(39) 



17 



•12 .7% 

(79) 

■10.8% 
(32) 



Adult 

-16.6% 

(63) 

-16.7% 
(44) 



*Numbers of exercises in parentheses 

Note: Minus percent figures represent median percentages 
balow national averages. 
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CHAPTER 3 



COMPARATIVE PERFORMANCE OF BLACKS, BALANCED, 
FOR OTHER MEASURED FACTOR^^^ 

Blacks are subject to many adverse conditions, and it is 
extremely difficult to identify the factors most responsible 
for the comparative disadvantage in science performance 
reported in the previous chapter. V7e can attempt to adjust 
our data for some of these factors through the application of 
"balancing" as explained in chapter 1 and appendix B. Many 
factors are not available in National Assessment data and we 
cannot, of course, adjust for these. Thus, the adjustments 
may well be too small, and therefore the balanced figures 
cannot be expected to represent the full potential in science of 
Black children and young adults under appropriate educational 
stimulation at home and in school. 

Blacks are underrepresented in the most educationally advan- 
taged type of community — the affluent suburbs — and are over- 
represented in the educationally disadvantaged inner city. 
(Results for different types of communities, showing the 
advantages and disadvantages , are presented in chapters 6 and 
7.) One might ask how much of the Black disadvantage could 
be accounted for in terms of this disproportionate represen- 
tation. Similarly^ thiice are disproportionately few Blacks 
whose parents have had the benef^its of completed high school 
and post high school education (see chapters 4 and 5) , and 
one might again ask how much the Black disadvantage would be 
reduced were Black and White levels of parental education 
balanced. A further factor involving disproportionate repre- 
sentation of Blacks is that of region (see chapter 8) . 

Exhibit 3-1 shows the distribution of relative performances 
by Blacks at all four age levels, b alanced for disproportionate 
type of community, parenta l educatTon, sex and regional 
representation . (For the conceptual basis of the balancing 
operation, the interested reader is referred to chapter 1, and 
for the arithmetic details, to appendix B.) Following the 
balancing adjustment, the median results for Blacks are still 
below national results, but by amounts that are substantially 
less than those which do not take the balancing factors into 
account. The residual disadvantages are -10.2%^ -11.0%, -7.7%, 
and -10.9% at ages 9, 13, 17, and adult, respectively. 
(Compared to -14.5%, -15.0%, -11.8%, and -15.8% as noted before 
in exhibit 2-1 . ) 
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Exhibit 3-1 

Distributions of Relative Performances of Blacks, Balanced, 
for all Science Exercises at Four Ages 



-p 


» — . 


o 




<D 




U 


c: 




o 


o 


•H 




4J 


dp 




rH 






> 




o 


o 






fU 






















KM 










rH 


O 




cu 


a 




o 


u 


'H 


o 


4J 


u 


rU 












:^ 




o 


o 


rH 




<u 


rH 









!5: 

m. 
s. 
0. 
-s. 
- ra. 

-IS. 
-20. 

-as. 

-3E3. 
-3S. 






No. of Exercises 
Median 



145 

-10.2% 



122 

-11. 0% 



124 
-7.7! 



119 

-10.9% 



12 



The 17~year-'Old age level retains the smaller disadvantage 
noted in the previous chapter. Other patterns over exercises, 
such as the difference between objective 1 and objective 2 
exercises remain in force after balancing. There is a general 
reduction in the Black disadvantage, age by age, and exercise 
by exercise. Exhibit 3-2 displays the relationship over 
exercises between unadjusted and balanced results for each age. 
It is clear in general that on the exercises for which Blacks 
perform well before balancing, they also perforin well after 
balancing . A similar statement could be made for exercises 
with relatively poor Black performance. 

Atypical Exercises 

The procedure for identifying specific exercises on which 
Black performance (after balancing) is atypically high or 
atypically low is identical to the procedure used in National 
Assessment Report 4 for sex, region, and size of community 
groups. On the distributions of relative performances for all 
exercises, upper and lower cutoff values are placed (see arrows 
on exhibit 3-1) in order to select and discuss exercises for 
which the Black disadvantage disappears, on the one hand, or is 
much worse than typical, on the other hand. The details of the 
procedure for selecting cutoffs are given in appendix C. 

9-Year-Olds 

The exercises for which Blacks perform atypically at age 9 
are listed in exhibit 3-3. Exercises labeled "R" have been 
released to the public (see National Assessment Report 1) and 
therefore, their content may be discussed in detail. Exercises 
labeled "U" , however, are being retained for reuse 
in later assessment cycles and therefore are not published at 
this time. In exhibit 3-3 and throughout this report, no summary 
phrase in the "content" column is given for these . unreleased 
exercises . 

In the upper section of exhibit 3-3 are listed exercises 
on which Blacks performed atypically well. Most of these 
exercises proved difficult for the national sample as a whole. 
Thirteen of the sixteen exercises show national percent correct 
of 36% or below, with seven of these thirteen having correct 
percentages below 20%. The 20% figure is what one would antic- 
ipate for a multiple-choice exercise answered by guessing at 
random among five alternatives, and therefore one may characterize 
national performance on these seven exercises as even worse than 
what respondents could obtain by guessing. (The situation 
actually is more complicated, since some percentage of respon- 
dents answered each exercise with "I don't know.") Altogether 
there are eight exercises at age 9 with national percentages 
below 20%, of which seven appear in exhibit 3-3. An eighth 
one shows a Black difference of only -3%. These very difficult 
exercises do not turn out to show substantial differences 
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Exhibit 3-2 



Relationship Between Unadjusted and 
Balanced Results for All Exercises 
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Exhibit 3-3 



Exercises Showing Atypical Effects, Balanced 
for Blacks, Age 9 

% Correct 



Exer- 






% Dif- 




cise # 


Black . Nat 


' 1 f erence 


Content 


U652* 


35 


33 


2 




R141* 


9 


7 


2 


Mixing water of 50° and 7 0° yields 










water of 60° 


U673* 


12 


11 


1 




U650* 


37 


36 


1 




U655* 


15 


14 


1 




U678* 


27 


26 


1 




U657* 


12 


11 


1 




U621* 


76 


76 


0 




U656* 


^12 


12 


0 




R14 0* 


14 


14 


0 


Dead plants form coal 


R142* 


96 


96 


0 


Balancing a* beam with one weight 


U617 


78 


78 


0 


U677* 


33 


33 


0 




R158* 


22 


23 


-1 


What is a scientific theory 


U671 


39 


31 


-1 


U672* 


18 


19 


-1 




U632^ 


47 


66 


-19 




U665* 


51 


70 


-19 




R147* 


49 


68 


-19 


To test an idea , try it 


U624* 


54 


73 


-19 


U629* 


50 


69 


-19 




U675* 


56 


75 


-19 




U633* 


46 


65 


-19 




R120* 


53 


72 


-19 


Transfer of momentum between objects 


U634* 


44 


63 


-19 


R109 


66 


86 


-20 


Rocks on Earth ' s surface are solid 


U618* 


57 


77 


-90 




R152 


33 


53 


-20 


Because of vaccinations few people get 










smallpox 


R150* 


35 


55 


-20 


From chart--sodium is least common in 










human body 


U667* 


43 


63 


-20 




U616* 


60 


80 


-20 




U661 


67 


87 


-20 




U611* 


67 


87 


-20 




U639* 


33 


54 


-21 




R123* 


44 


66 


-22 


What scientists learn from fossils 


R116* 


57 


79 


-22 


Mushrooms do not have green leaves 


U668* 


41 


63 


-22 


*Exercises 


also 


identified 


as atypical before balancing. 


Note : 


The 


line separates atypically high from atypically 



low exercises. 
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between groups that differ on most other exercises. 



The difficult exercises are often tricky and even misleading 
to respondents at the given age. For exaimple, exercise R141 
is as follows: "A pint of water at a temperature of 50^ 
Fahrenheit is mixed with a pint of water at 7 0° Fahrenheit. 
The temperature of - the water just after mixing will be about: 
( )20° F.; ( ) 50° F.; ( ) 60° F.; ( ) 70° F.; ( ) 120° F.; 
( ) I don't know." Nine-year-olds are apt to be confused by the 
references to Fahrenheit, pints and the notation. Most 
(69%) treated the problem as one requiring addition of the two 
numbers, and responded 120° F. Of the national sample, 12% 
responded "I don't know," and only 7% responded with 6 0^ F., 
the correct answer. Such difficult exercises do not tend to 
differentiate v/ell between different groups of respondents, in 
particular between Blacks and non-Blacks. About all one can say 
is that on such exercises. Black and non-Black respondents are 
about equally often misled. 

There are a couple of exercises where other factors may be 
at work behind the relatively good Black performance level. 
Exercises U621 and U617, both unreleased, require information 
about nutrition and volcanoes, respectively. The particular 
information involved is quite standard in first- and second- 
grade curricula, and the lack of color disparity is perhaps not 
too surprising. A third exercise, R142, requires the children 
to level a balance by hanging a weight appropriately on the 
lighter arm and reading off the proper numerical indicator on 
the arm. This was a tr ial-and-error exercise with actual weights 
rather than a paper-and-pencil question, and on such a directly 
experiential task. Black children did as well as White children. 
In fact, both groups exceeded 96% success. 

The result from this last exercise hints that Black per- 
formance at the youngest age level is improved for exercises with 
direct experiential content. Furthermore, the Black disadvan- 
tage is great for exercises far removed from daily experience. 
The lower section of exhibit 3-3 lists the exercises with 
atypically low Black performance. Four of the unrele^csed 
exercises involve relatively remote textbook facts about 
astronomy, one about geology, one about meterology, two about 
chemistry, and one each about electricity, human biology and the 
behavior of animals. Two call for using symbols and graphs, 
and one refers to the activities of scientists. The released 
exercises in exhibit 3-3 convey the specific flavor of the kinds 
of questions on which Black 9-year-olds perform most poorly 
relative to other 9-year-olds. Exercise R116, with a Black 
disadvantage of 22%, reads, "Which of these plants does NOT 
have green leaves: ( )A dandelion; ( )Grass; ( )A mushroom; 
( )A willow tree; ( )I don't know." Another exercise (R123), 
with a disadvantage of 22%, is, "What can scientists learn by 
studying fossils: ( ) Why earthquakes took place; ( )What 
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animals lived long ago; ( ) How far the moon is from Earth; 
( )What the weather will be tomorrow; ( )I don't know." This 
exercise, besides covering a content area quite removed from 
daily experience, requires that the children understand the 
vocabulary item "fossils." Other released exercises with a 
notable vocabulary component are R152, concerning vaccinations, 
on which the Black deficit is 20%, and R109, which refers to 
the Earth's "surface" (deficit also 20%). 

Two other released exercises listed in exhibit 3-3, R147 and 
R120, per-v'-.ain to rather disparate contents: how the respondent 
might best tell if salt water taffy is really made just by 
mixing water, ^alt ar*d sugar (mix tho>5e things and see what 
happens); and what would happen to a wooden block hanging on a 
string if another such block were released from the left in an 
arc to hit it {it will swing off to the right) . These two 
exercises both require an abstract attitude toward unusual uses 
of familiar objects — a detached "What If" approach. If such an 
attitude is indeed needed to achieve correct responses on these 
two exercises, it apparently does not come as easily to Black 
9~year~old respondents . 

13-¥ear-01ds 

Exhibit 3-4 lists the exercises on which Blacks perform 
atypically at age 13. Of these fifteen exercises on which 
Blacks perform atypically well, five have national percent 
correct figures below 20%, and another three between 20 and 
40%. Thus there is some indication that difficulty undifferen- 
tiating exercises occur at the IS-year-old as well as the 
9-year-old level. One or two exercises in this group, however, 
deserve note as being different from the difficult science 
achievement questions discussed for 9-year"olds . Released 
exercise R24 7, with a Black advantage of 3%, is, "Do you ask 
questions about why things in nature are the way they are: 
( )Often; ( )Sometimes; ( )Neviir...If so, what questions have 
you asked most recently?" An "often" questioning attitude is 
reported by a slightly greater proportion of Black than non-Black 
children. Unreleased exarcise U775 is similar in its import. 
Whether this reported curiosity is specific to science is not 
clear • 

Several of the exercises on which age 13 Blacks do about as 
well as non-Blacks have reference to direct experience. 
Exercise R217 is, "Most of the chemical energy of the gasoline 
burned in a car is not used to move the car but is changed into: 

( ) electricity; ( )heat; ( ) light; ( )magnetism; { ) sound; 

( )I don't know." Answer: heat. Exercises R202 and R203 ask the 
purpose of toothbriAShing , and what kind of clouds bring rain. 
Two unreleased ex^^rcises, 0729 and U767, are questions based on 
everyday experiences about human biology and about scientists, 
respectively. In R237f the respondent is given a stopwatch and 
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Exhibit 3-4 



Exercises Showing Atypical Effects, Balanced, 
for Blacks , Age 13 

% Correct 



Exer- 






% Dif- 




cise # 


Black 


Nat ' 1 


ference 


Content 


R247* 


10 


7 


3 


Do you often question things in 










nature 


U774* 


20 


19 


1 




U747* 


8 


8 


0 




U741* 


34 


34 


0 




U775* 


5 


5 


0 




R222* 


38 


38 


0 


By natural selection, why giraffes 










have long necks 


U769* 


66 


66 


0 , • 




U76 7* 


89 


89 


,0 • 




R237* 


34 


35 


-1 


Time a pendulum's swings 


R243* 


67 


68 


-1 


Measurements in science : close not 










identical 


U729* 


50 


51 


-1 




U766 


9 


11 


-2 




R217* 


46 


48 


-2 


Burning gasoline in a car creates 










heat 


R202* 


96 


98 


-2 


Why brush your teeth 


R203* 


90 


92 




Thick , dark clouds bring ra in 


U709* 


61 


80 


-19 




U750* 


60 


79 


-19 




R241 


60 


79 


-19 


Mathematics a useful skill in 










science 


R234* 


43 


62 


-19 


Balance beam — weight in pan ; weight 










on hook 


U754* 


-4 4 


63 


-19 




R205* 


66 


85 


-19 


Comfortable temperature : 7 0 °F 


R236* 


40 


60 


-20 


From chart--determine dog ' s food 










3;ation 


R212* 


38 


59 


-21 


Movement of air masses predicts 










/eather 


U707* 


63 


84 


-21 




R214* 


33 


54 


-21 


Jounterbalancing unequal weights on 










beam 


R206* 


57 


78 


-21 


Why fan a campfire 


U717* 


41 


63 


-?2 




U760* 


19 


42 


-23 




U758* 


26 


49 


-23 




U759* 


23 


47 


-24 




U727* 


27 


53 


-26 
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Exhibit 3-4 (Continued) 



% Correct 



Exer- 






% Dif- 




cise # 


Black 


Nat' 


1 ference 


Content 


U753* 


43 


69 


-26 




R213* 


30 


57 


-27 


Radio waves least upset plant and 










animal life 


R233* 


43 


70 


-27 


From chart--compare guinea pig 










weights 


R215* 


24 


52 


-28 


Flower seeds develop from ovules 



^Exercises also identified as atypical before balancing. 

Note: The - - -line separates atypically high from atypically 
low exercises. 
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asked to time ten swirxgs of a pendulum. 

When we turn to those exercises on which Black IS-year-olds 
perform especially poorly in the lower section of exhibit 3-4, 
there is evidence of the same bookish and abstract qualities 
noted earlier for poor performance of Black 9-year~olds. Among 
released exercises, there is a vocabulary component for R215, 
where the correct answer is "ovules"; R205, where the most 
comfortable "Fahrenheit" temperature is solicited; and R212, 
where information was asked about predicting weather, with the 
correct answer, "The movement and characteristics of air masses." 
Black deficits on these exercises, are 28%, 19%, and 21%, 
respectively. Unreleased exercises U707, U717, and U727, with 
comparable deficits, involve terms that may also be relatively 
unfamiliar to Black children. 

Another set of exercises with poor Black performance involves 
the plotting and interpretation of data graphs . On exercise 
R236, the child is shown a month-by-month chart of the weight 
of a dog. He is given recommended directions for feeding a dog 
according to its weight, and must then specify what food ration 
is appropriate in a given month. On exercise R233, a study of 
the daily weights of two guinea pigs is described, and the 
respondent must pick the verbal statement which corresponds 
accurately with a graphical sumnary of the results. Unreleased 
exercises U758 and U760 require the child to use physical appa- 
ratus to generate data which is to be graphed, and then to label 
and interpret the graph. On all four of these exercises, the 
deficit exceeds 20%. Another two exercises involve anticipation 
or calculation of the appropriate way to balance a beam. On 
exercise R214 (19% below national) , the child must interpret a 
sket^ch of a lever arm and apply the principle that a 2-lb. 
weight four units from the pivot will balance a 4-lb. weight two 
units away on the other side of the pivot. On exercise R234 
(21% below national) , the respondent is given a balance pan and 
some weights, along with a pictorial representation of this 
apparatus, and is asked three questions about balance points and 
weights. On the first question, performance of Black 13-year- 
olds-^as atypically poor (19% below national) . 

The remaining exercises in the lower section of exhibit 3-4 
represent an assortment of contents. Exercise R213 poses an 
ecologi.cal question in which the respondent must pick the one 
of five given actions which will not upset nature's balance, 
the""^rrect choice being the broadcasting of radio waves. 
Exercise R206 inquires why fanning helps campfires, and R241 asks, 
"Which of the following is most useful in scientific research: 
( )music; ( )magic; ( )marketing; ( )mathematics; ( )manu~ 
facturing." The unreleased exercises not thus far discussed 
'include U709, which deals with astronomy; U754, posing a 
hypothetical measurement problem; U750 and U753, exploring the 
difference between a "fact" and an "opinion"; and U759, which 
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involves a rough numerical estimation. 

In general. Black 13-year-olds do most poorly relative to the 
national sample on exercises involving either unfamiliar terms or 
remote content materials for which the correct approach involves 
the detached indirection characteristic of scientific method . " 



IV-Year-Olds 



At the 17-year-old level, there is a reappearance of the 
same factors accounting for atypically good and atypically poor 
performances among Blacks at other age levels (exhibit 3-5) . 
In the upper group of exercises, we find a few difficult, 
undifferentiating exercises (U835, R332, U843) . Unreleased 
exercise U841 is also quite difficult, but appears to differen- 
tiate groups in that Black respondents perform about 17% better 
than national average, after balancing (and also before 
balancing) . Since this is by far the greatest Black advantage 
on any science exercise in National Assessment, one would like 
to be able to point to some special content factor involved in 
this exercise. Unfortunately, it is likely that fortuitous 
features of the format are responsible for the unique charac- 
teristics of this exercise. Several graphical alternatives 
are offered as completions for a question on a physical 
equation. The exercise is of such difficulty that respondents 
probably chose among the graphs on the basis of their superficial 
appearances . 

Exercise R322 is also a tricky one. The correct alternative 
is the one which completes the following seemingly paradoxical 
sentence: "If a person who is a light eater has a tendency to 
be overweight, it is most likely due to highly efficient 
utilization of food by the body." We do not know what fraction 
of the national 48% correct is due to lucky misunderstanding, 
such as reading " inefficient" for "efficient" and thinking this 
to lead to overweight. Therefore, any interpretation of the 
Black advantage of 8% is hazardous. 

Three exercises of interest yield very slight Black advan- 
tages after balancing, but do not exceed the upper cutoff and 
are therefore not shown in exhibit 3*^5. One of these is 
released exercise R35 0 . It reads , "Whenever scientists carefully 
measure any quantity many times, they expect that: ( )all of 
the measurements will be exactly the same; ( )only two of the 
measurements will be exactly the same; ( )all but one of the 
measurements will be exactly the same; ( )most of the measure- 
ments will be close but not exactly the same; ( )I don't know." 
The balanced proportion of Black 17 -year-olds choosing the 
correct and eminently sensible answer (most will be close but 
not exactly the same) was 2.7% above the national average. An 
unreleased question (U8 07) about physical principles in home 
appliances yields a balanced Black advantage of 1.6%, and a 
released exercise (R3i4) about why giraffes evolved long necks, 
a balanced Black advantage of 1.5%. The latter is the same 
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Exhibit 3-5 





Ex 


ercises 


Showinc 


' Atypical Effects, Balanced, 








for 


Blacks, Age 17 




% Correct 






Exer- 






% Dif- 




cise # 


Black 


Nat' 1 


ference 


Content 


U341* 


J D 


1 Q 


17 




R32 2* 


Z> D 


A Q 

4 o 


8 


Efficient use of food can cause 










overweight 


U8 35* 




2 b 


1 




•D O O O * 


z4 


Z 0 


A 
*i 


How scientists determine rock age 


U O 4 J 


1 o 


15 


J 


u856* 


4 2 


60 


-18 




Uo 5 5* 


43 


62 


— 19 




U813* 


48 


67 


-19 




US 11* 


52 


71 


-19 




R312* 


40 


59 


-19 


Unaided eye detects certain wave- 










lengths of light 


D809* 


51 


72 


-21 




U816* 


37 


60 , 


-23 




R316* 


29 


53 


-24 


Adrenalin is a stimulant to the heart 


P309* 


41 


65 


-24 


Who proposed natural selection in 










evolution 


R305* 


52 


76 


-24 


Movement of air masses predicts 










weather 


R344* 


30 


55 


-25 


Time a pendulum ' s swings 


R340* 


54 


79 


-25 


Which weight experiment gives strongest 










evidence 


R341* 


49 


74 


-25 


Balance beam — weight in pan; weight 










on hook 


U820* 


29 


56 


-27 




U852* 


40 


73 


-33 





*Exercise3 also identified as atypical before balancing. 

Note: The- - -line separates atypically high from atypically 
low exercises. 
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exercise as R222 for 13--year-olds , on which the balanced Black 
advantage was 0%. 

In the lower group of exercises (exhibit 3-5) , vocabulary 
factors again account for some of the re'sults. Released 
exercise R316 requires the knowledge that "adrenalin stimu- 
lates the heart," and unreleased exercises U&09 and U820 also 
use unfamiliar words involving aspects of body chemistry. 
Exercise R309 as^ks for the association of Darwin's rame with 
the idea of natural selection in a theory of evolution . 
Exercise R305 is the same question as R212, about "movement 
and characteristics of air masses," on which Black 13-year-olds 
also showed an atypically great deficit. 

Other exercises replicating poor 13-year-old results are: 
U856, which like U7 58 required a graph of data generated by 
the respondent from a physical apparatus; and R341, which like 
R234, gave the respondent a balancre pan, weights and pictorial 
instructions and asked how to ^balance a weighted pan. A 
further question with the same apparatus is unreleased exer- 
cise U852, with a 17-year-old Black balanced deficit of 33%. 
Clearly, Black 17-year-olds experienced some difficulty with 
laboratory exercises (although the number of 17-year-old Black 
respondents exposed to each r^uch exercise was small, about 80^ 
and therefore the percentage results are statistically 
unstable) . Another laboratory exercise giving an atypically 
large deficit is R344, involving the timing of ten swings of 
a pendulum. This particular exercise showed no Black disad- 
vantage in its age 13 form (R243) . Comparing the percentages 
of correct responses on the pendulum exercise for the two ages 
in exhibits 3-4 and 3-5, it appears that there was no Black 
improvement from ages 13 to 17, but a sizasble national improve- 
ment (about 20%) . It is not clear whether this result indi- 
cates that the progress of Black teenagers suddenly stops on 
certain laboratory science tasks, or whether there is some 
fortuitous peculiarity in the administration of this particular 
exercise. Exercise R340 presents hypothetical experiments 
involving the measured weights of two objects, with four 
observations on each weight in each experiment. On this exercise, 
there is a Black deficit of 25%. 

The other five exercises in exhibit 3-5 include R312 , a 
released exercise comparing different forms of electromagnetic 
radiation, with the answer that only "certain wavelengths of 
light" can be detect-.ed with unaided human eyes. The unreleased 
exercises (U855, U811, U813, and U816) cover various topics in 
electricity^ the mechanics of springs, properties of light, * 
and the concept of a hypothesis. 

Adults 



At the adult level, exhibit 3-6, six exercises show 



Exhibit -6 



Exercises Showing Atypical Effects,. Balanced, 
for Blacks, Young Adults 

% Correct 



Exer- 






% Dif- 




cise # 


Black, Nat'l 


ference 


Content 


U968* 


56 


45 


11 




R433* 


24 


15 


9 


How scientists determine rock agG 


U948* 


14 


9 


5 




R410* 


66 


62 


4 


Current involves movement of electrons 


R418* 


55 


51 


4 


Speed of falling rock increases 


U921* 


67 


63 


4 


R405* 


44 


70 


-26 


Secondary result of vasectomy" 


R406* 


43 


S9 


-26 


Adrenalin is a stimulant to the 










heart 


U919 


37 


64 


-27 




R403* 


57 


84 


-27 


Movement of air masses predicts 










weather 


U957* 


25 


54 


-29 




U956* 


23 


56 


-33 




*Exercises 


also identified 


as atypical before balancing. 


Note : 


The- 


- -line 


separates atypically high from atypically 



low exercises. 
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atypically good Black performance; of these, two {R433 and U948) 
are so difficult as to be undifferentiating. Of the remaining 
four, the two released ones are as f ollows--R410 : "An electric 
current in a copper wire involves mainly the movement of: 

( ) copper atoms; ( ) copper molecules; ( ) electrons; 

( )neutrons; ( ) protons; ( )I don't know." and R418: "A 5-pound 
rock is dropped from a cliff 500 feet high. The longer the 
rock falls, the greater is its: ( ) acceleration; ( }potential 
energy; ( ) speed; (')total energy; ( ) volume; ( )I don't know." 
The Black advantages, after balancing, on these two exercises 
are each 4%. Both have the characteristic that the straight- 
forward answers are correct — "electrons" for the firs't, and 
"speed" for the second exercise. Black adults tend slightly 
less often than non-Black adults to be tempted to choose the 
fancy incorrect alternatives. The remaining two exercises with 
atypically good Bl^ck performance are both unreleased, U968 and 
U921. Exercise U968 is an attitude question (objective 4) 
emphasizing curiosity in everyday matters, and U921 is a 
question about astronomy verifiable by direct observation. 

Adult exercises with atypically poor Black performance 
include two released questions overlapping with 17-year-old 
exercises with atypically poor Black performance. R406 concerns 
adrenalin as a stimulant to the heart, and R403 the importance 
for weather prediction of the "movement and characteristics 
of air masses." Two unreleased adult exercises also overlap 
with previous exercises and yield comparable results. On 
exercise U956, which repeats U760 and U856 involving the 
graphing of laboratory data, there is a Black deficit of 33%. 
The follow-up graphical exercise U957 shows a deficit of 29%; 
the parallel* question at age 13, U758, yielded a deficit of 
23%. Unreleased exercise U919 overlaps U813 concerning 
properties of light, and at both ages 17 and adult there is 
atypically poor Black performance. The only newly appearing 
exercise in the lower portion of exhibit 3-6 is R405/ with 
Black adults performing 26% less well than national, after 
balancing. This exercise concerns the consequences for adult 
human males of tying off their main sperm ducts in a simple 
operation. The alternative answers were: "( )The voice will 
eventually become high pitched; ( ) Fatty pads will gradually 
develop on the hips; ( ) Behavior will eventually become more 
effeminate; ( )The hair v/ill develop in longer strands than 
usual; ( )None of the above will occur; ( )I don't know." The 
question as formulated requires the answer "None of th^^above," 
and as such is perhaps more an assessment of a non-anxious 
attitude toward side effects of apparently sensitive medical 
procedures than a direct probe of the knowledge of the purpose 
of vasectomy. 



In gener^, over all age levels of what is probably the most 



Summary 
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suggestive theme running through these atypical exercises, we 
may state the following : Blacks perform best on those science 
exercises most dependent upon daily experience and common 
knowledge and poorest on those which involve a detached research 
attitude toward the objects and phenom<:^na of science . There 
are several factors m the Black educational , environmental , 
and cultural situation that might explain such an outcome. It 
is not possible from a single cycle of National Assessment to 
choose among the many conceivable sets of explanatory factors. 
A number of research efforts over a period of years would be 
needed to begin to resolve these questions. If the science 
performance of Black youngsters improves steadily in the years 
ahead, however , the need to explain the def icit on the first 
National Assessment will recede in importance . We all hope 
that some of the many current innovations in methods and 
technology of teaching and in the organization of educational 
systems will produce substantial educational gains for all 
children , Black and non-Black . Future National Assessment 
cycles will show the pattern and extent of any such improve- 
ments. 



\ 

\ 



CHAPTER 4 



PARENTAL EDUCATION GROUPS 



Every respondent to NAEP exercises, at age levels 9, 13, 
17, and adult, was asked to indicate the highest year of 
schooling completed by his mother and his father. In this 
chapter, we discuss the percentages of correct responses to the 
science exercises for each of several levels of reported parental 
education : 

Level Definition 

0 Education of parents was not ascertained . 

1 Neither mother nor father was educated beyond 
8th grade. 

2 Either mother or father had some high school, 
but neither completed high school. 

3 Either mother or father completed high school, 
but neither was trained beyond high school . 

4 Either mother or father was educated beyond 
high school. 

It should be stressed that the data available on education 
of parents (except as noted below for some 9-year-olds) is that, 
reported by the respondent on the "tail-sheet" of the exercise 
booklet . If the respondents were in error in reporting the 
schooling of their mothers or fathers, they will be misclassi- 
fled on parental education. 

More often than at other ages , 9-year-olds failed to respond 
to the questions about education of their parents. In some 
cases where responses to these questions were omitted, exercise 
administrators provided the information from school records. In 
many other cases, the parents' education was not ascertained, 
and the respondents were classified at level 0 on parental 
education. 

The median number of respondents classified at every level 
of parental education for each age is shown in exhibit 4-1. 
(For different boooklets or "packages:'' of exercises, the actual 



Exhibit 4-1 



Median Numbers of Respondents Per Exercise 
for Each Level of Parental Education 



Level of parental education 



Age 0 1 2 3 

9 789 125 144 567 

13 252 109 250 762 

17 39 146 347 697 

Adult 51 293 159 200 
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number of respondents varies somewhat, above and below these 
median numbers.) It may be seen from exhibit 4-1 that, for many 
9^year-olds, the information was not ascertained. Also, it is 
likely that at age 9, more than at the other ages, respondents 
made errors in responding concerning the' numbers of years of 
school for their mother or father. 

Before presenting any detailed results, it is of interest 
to consider whether, on typical science exercises, performance 
differences are related to the reported level of parents* 
education. Exhibit 4-2 shows graphically the median effect for 
respondents at each level of parental education at each of the 
four ages. At each age it is apparent that median performance 
is below the national median for levels 0 (unascertained) , 1 (no 
high school) , and 2 (some high school) , rising above the national 
median for level 4 (some education beyond high school) . At level 
3 (graduated from high school) median performance is near the 
national median. On typical science exercises, respondents who 
report that at least one parent had substantial formal schooling 
perform better , as a group , than respondents who report less 
schooling for both parents. 

Throughout this chapter, we must remember that respondents 
classed at a given level of parental education may differ in 
many ways from those classed at other levels. In particular, 
the various levels of STOC (size and type of community--see 
chapter 6), sex, region, and color may not be proportionately 
represented at each level of parents' education. Effects that 
are apparently related to parental education, then, in some 
cases may be more simply explained by differences in STOC or 
differences in color that are masquerading as differences in 
education. In chapter 5 we shall return to this matter. For 
now, we simply present the data by level of parents' education 
without any adjustment for the possible effects of other 
variables . 

Level 1: Neither Parent Attended High School 

Exhibit 4-3 shows the distribution of effects at parents' 
education level 1 for all science exercises at each age. At 
each age, the median percent correct is below that for the 
national sample. The median effect ranges from -7.2% at age 9 
to -11.8% at. age 13. This tells us that, for the average 
science exercise, respondents whose parents failed to attend 
high school obtain correct answers from 7 co 12 percent less 
often than the national average for each age group. 

The arrows in exhibit 4-3 show the "cutoffs" beyond which 
exercises are considered atypical in their effect (for defini- 
tion of "cutoff" see appendix C) « Some exercises are atypical 
in showing greater than usual effects of level 1, parents' 
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Exhibit 4-2 

Median Effects for Levels of Parental Education at Four Ages 




Exhibit 4-3 



Distributions of Effects for No High School Parental 
Education for all Science Exercises at Four Ages 




education. Other exercis:=»s are atypical in showing less^.:r than 
usual effects^ or effects in the direction opposite to the 
usual directioi:. For a few science exercises^ respondents 
reporting that neither parent attended high school give correct 
responses at a rate higher than the national percent correct. 
The discussion of atypical exercises is contained in chapter 5. 

Level 2: At Least One Parent Attended High School 

In exhibit 4-4 appear the distributions . of effects for level 
2 of parental education for all science exercises . At each age , 
median performance of respondents for whom neither parent 
completed high school is below the median performance for the 
nation as a whole ^ and the majority of all science exercises 
display the effect. The effect^ however, is less marked for 
adults than for the other three age groups. Exercises that 
show atypical effects fall in the extremes of the displays of 
exhibit 4-4, beyond the cutoffs shown by arrows. 

Level 3; At Least One P arent Graduated from High School 

Exhibit 4-5 shows that, on most exercises, performance of 
respondents at parents' education level 3 is similar to that for 
the national sample. At age 13, there is a slight, tendency 
for such " respondents to perform less well than the average for 
a3.1 13-year-olds, while at the adult age, such respondents 
perform a bit better than the average for all adults. Beyond 
the arrows in exhibit 4-5 are only a few exercises for which 
effects of education level 3 are atypical. 

Level 4; At Least One Parent Educated Beyond High School 

Exhibit 4-6 displays the distribution of effects for 
respondents at parental education level 4. Median effects 
range from 5.1% at age 17 to 9.1% for adults. At each age, 
the body of results attests to the higher performance on 
science exercises of those respondents with at least one parent 
with post high school education. A relatively large number of 
science exercises show atypical level 4 effects. These are the 
exercises that fall beyond the arrows in exhibit 4-6. Those 
that remain atypical for balanced results are presented and 
discussed in chapter 5. 

Level 0; Parental Education Unascertained 

From exhibit 4-7, it is clear that for respondents who fail 
to provide the requested information about their parents ' 
education, performance on science exercises tends to be below 
the national average. The median effect ranges from -4.6% at 
age 9 to -16.8% for adults. The relatively small deficit at 
age 9 suggests that all levels of actual parental education 
may be represented in this group of children who are unable to 
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^Ixhibit 4-4 

Distributions of Effects for Some High School Parental 
Education for all Science Exercises at Four Ages 
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say how far their parents continued in school. But the story 
is quite different for adults. Indeed, for adults at level 0, 
performance is lov.^er than for adults at level 1 (neither parent 
attended high school) . 

Parental Education Effects for Various Science Ob j ectives 
an d for Ph y sical Science Versus Biological Science 

Median relative performance for the various levels of 
parental <^ducation are given separately for the four science 
objectives in exhibit 4-8. It may be seen that the trend showing 
better performance with increasing levels of parental education 
appears to hold separately for each science objective at each 
of the four ages. In general, the trend is quite similar for 
the exercises designed to assess the various science objectives. 
There is some suggestion of smaller effects associated with 
parental education for objective 4 (attitudes towards science 
and scientists) , although the smaller numbers of exercises 
administered for this objective prompt us to be somewhat 
cautious about this finding. 

In exhibit 4-9, educational effects are displayed separately 
for physical science exercises, biological science exercises 
and unclassified exercises. The trend of effects dependent 
upon parental education is essentially the sane for all three 
classes of exercises. 
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Exhibit 4-7 

Distributions of Effects for Unascertained Parental 
Education for All Science Exercises at Four Ages 




Exhibit 4-8 



Median Effects Associated With Parental Education Groups 
for Science Exercises Classified by Objective 



Level of parental education 



*1 - Facts and principles 

2 - Abilities and skills 

3 Understand investigative nature 

4 - Attitudes and appreciations 
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Age 


Objective* 


0 


1 


2 


3 


4 


9 


1 


-4.4 


-7.3 


-4.6 


0.7 


5.4 




2 


-6.2 


-6.7 


-3.5 


0.8 


6.9 




3 


-4.5 


-7.1 


-7.3 


0.1 


6.2 




4 


-3.7 


-9.0 


-5.4 


-0.4 


4.9 


13 


1 


-7.1 


-11.5 


-6.2 


-1.5 


5.4 




2 


-10.4 


-12.5 


-7.5 


-1.4 


5.7 




3 


-11.0 


-13.5 


-3.3 


-1.3 


4.9 




4 


-4.4 


-2.0 


-2.6 


-0.2 


1.4 


17 


1 


-7.7 


-8 . 0 


-7.1 


0.2 


4.9 




2 


-12. 5 


-10.5 


-9.7 


0.0 


.3 




3 


-4.5 


-6.3 


-3.9 


-0.8 


4.0 




4 


-5.0 


-1.7 


-6.3 


0.1 


4.6 


Adult 


1 


-15. 6 


-8.2 


-1.2 


2.4 


9.2 




2 


-18. 1 


-7.2 


-5.2 


5.9 


6.2 




3 


-17.1 


-10.6 


-5.6 


2 . 0 


11.2 




4 


-11.2 


-5.3 


-.4 


0.0 


5.4 
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Exhioit 4-9 



Median Effects Associated With Parental Education 
Groups for Physical Science , Biological 
Science, and Unclassified Exercises 



Level of parental education 



*P - Physical Science 
B - Biological Science 
U - Unclassified Exercises 
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Age 


P, B, or U* 


0 


1 


2 


3 


4 


9 


P 


-4.5 


-9.4 


-4.4 


0.5 


6.2 




B 


-4.4 


-5.9 


-3.9 


0.8 


5.0 




U 


-4.7 


-7.1 


-8.1 


-0.3 


7.0 


13 


P 


-7.4 


-11.1 


-6.4 


-1.4 


5.2 




B 


-7.8 


-12.2 


-6.2 


-1.9 


5.4 




U 


-9.2 


-12.3 


-3.6 


-0.9 


3.9 


17 


P 


-7.2 


-9.4 


-8.4 


0.1 


5.3 




- B 


-12.6 


-6.5 


-7.2 


0.7 


4.9 




U 


-5.0 


-7.6 


-6.3 


-0.1 


4.6 


Adult 


P 


-15.6 


-7.6 


-2.5 


3.7 


7.6 




B 


-17.8 


-8.7 


-0.8 


2 . 0 


10.2 




U 


-16. B 


-7.0 


-2 . 8 


0.3 


11.0 



standardized Effects for Parental Education 



So far, the results for parental education groups have 
been stated in our customary way--as advantages or deficit 
compared to national. We have noticed that, as would be expected, 
parents of school children tend to have gone further in school 
than parents of young adults . This fact makes it impossible 
for the advantages and deficits of the various education groups, 
compared to national performance at their age , to be the same 
at all ages. Systematic differences between the curves of 
exhibit 4-2 are inevitable once we know the median number of 
respondents (exhibit 4-1) . 

The simplest way to obtain numbers for which this feature 
is kept from confusing us is to compare the groups using a some- 
what different standard . Instead of using the actual national 
percent of success as a standard, let us use the percent success 
for a standard population in which: 

2 5 percent of parents have no high school 
25 percent of parents have some high school 
25 percent of parents graduated high school 
25 percent of parents went beyond hign school 

(This is, incidentally, about what happened for parents of young 
adults) . Results expressed in terms of this new standard we 
shall call standardized results. 

The arithmetic is simple. At age 9, for instance, the 
performances compared to national were, before standardizing: 



grade school -7.18 

some high school -4 • 76 

graduated high school . 5^ 

beyond high school 5 . 83 

Subtracting the mean of these four values (-1.39) we obtain the 
following standardized results at age 9: 



grade school -5.8 

some high school -3.4 

graduated high school 2 . 0 

beyond high school 7 . 2 

Another way to describe what we are now doing is to say 
that wo are comparing each parental education group with the 
mean of the four parental education groups . 



The result of doing this analysis for all ages is shown in 
exhi.bit 4-10. We see that, having eliminated the confusion ^,ue 
to chai jing degree of parental education , the behavior of each 
parental education group is almost the same at all ages. The 



median effects, to the nearest percent, are 

grade school -7% 

some high school -3% 

graduated high school +2% 

beyond high school +8% 

and no individual value differs from these medians by as much 
as 2%» 
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Exhibit 4-10 

Standardized Median Effects for Parental 
Education at Four Ages 
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CHAPTER 5 



PARENTAL EDUCATION GROUPS: BALANCED RESULTS 



The previous chapter presented results on science exercises 
for parental education groups as they are > Within the group 
characterized as level 1 on parental education (where neither 
parent attended high school) we have relatively heavy represen- 
tation of respondents from the inner city and from rural 
communities, of Blacks, and of Southeasterners . When we consider 
balanced effects for level 1 respondents, we are asking how 
their performance would be expected to compare with national 
performance if the proportion of inner city and rural respondents, 
the proportion of Blacksv and proportion of Southeaster aers 
v;ere the same in level 1 of parental education as for the national 
sample. The balanced results give a clearer impression of 
differences associated with parental education after removing 
effects that might be attributed to the classification of 
respondents by region, type of cortimunity or color. 

Exhibit 5-1 provides some infoirmation about changes in the 
results for science exercises as a consequence of balancing. 
We consider the difference between the medians at the highest 
and lowest levels of parental education for each of the four 
age groups. After balancing, that difference is reduced by 
between two and six percentage points. After balancing, the 
difference between median performance at educational level 4 
and at educational level 1 is relatively uniform over ages, 
ranging between about 9% and 12%. The conclusion is the same if^ 
we look at balanced results only for those exercises common to 
two or more ages. While some exercises show greater or lesser 
effects of parental education, the difference between median 
balanced performance of respondents who report that a parent was 
educated beyond high school and respondents who report that 
neither parent attended high school is roughly 10% at all 
age levels. 

LeL- us now turn to a more detailed perusal of results for 
each level of parental education. In particular, we consider 
science exercises with effects differing markedly from the 
overall median effect. 

Level 1: Neither Parent Attended High School 

For level 1, the balanced median deficit from the national 
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Exhibit 5-1 



Percentage Differences Between Median Effects at Parental 
Education Levels 4 and 1 for Unadjusted and 
Balanced Results at Each Age 



Age 


Unadjusted 
difference 


Balanced 
difference 


Change due to 
balancing 


9 


13. 0% 


9.1% 


3.9% 


13 


17. 0% 


11.2% 


5.8% 


17 


13.6% 


11.6% 


2.0% 


Adult 


17 . 0% 


12.4% 


4.6% 
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percentage correct varies from about 5% (at age 9 and at adult) 
to about 7.5% (at ages 13 and 17). However, at each age,, some 
science exercises show atypically greater effects and some 
atypically lesser effects. Data about these atypical exercises 
appear in exhibit 5-2. 

At age 9, two released exercises (R149 and R120) show level 
1 deficits of 19% and 22%, respectively. R149 requires scien- 
tific reasoning from a pictorial representation. At neither 
age 13 nor 17 are there released exercises that show an atypically 
large deficit at level 1. At the adult age level, R407 and 
R411 show large deficits, between 14% and 15%, whereas the 
median deficit is about 5%. Both require the mastery of know- 
ledge — i.e., that testes produce sperm, and that flower seeds 
are produced by ovules. 

Thebe exercises, and those unreleased exercises that show 
atypicJlly larc^^ level 1 deficits, seem to fall into two 
classes: either (1) scientific knowledge is required and 
relatively unfamiliar words must be recognized (e.g., ovules, 
testes , photosynthesis ) or ( 2 ) a cognitive task of translation 
from graphical or pictorial evidence to a conclusion stated 
in words is required. Respondents at this lowest parental 
education seem to have special difficulty with these two kinds 
of science exercises. 

Certain other exercises (R319 and R418) are atypically easy 
for respondents at level 1 of parental education; indeed, these 
re2Spondents obtain correct answers more often than the national 
average. . A careful review of the content of these exercises does 
not lead us to any simple hypothesis concerning why they appear 
to be unusually easy for respondents whose parents failed to 
attend high school. 

Level 2: At Least One Parent Attended High School 

Level 2 respondents tend to perform less well than the 
national average. The balanced median effects show deficits 
between 1% (for adults) and 5% (for 17-year-olds) . 

Inspection of exercises atypical at level 2 (exhibit 5-3 ) 
suggest that the same principle may be at work here as at 
level 1. Exercises requiring reasoning from graphs or pictures 
tend to be exceptionally difficult for respondents whose parents 
failed to complete high school, as do exercises that demand 
special vocabulary knowledge. 

Level 3: At Least One Parent Completed High School 

For respondents at this level , overall performance is close 
to the national average, with median balanced effects ranging 
from about -1% at age 13 to +1% for adults. Inspection of 



Exhibit 5-2 



^•Exercises Showing Atypical Effects, Balanced, 
for Education Level 1 

(Neither Parent Attended High School) ' 
% Correct 





Exer- 


Level 




% Dif- 






cise # 


1 


rja u 


1 ference 


Content 


9 


U621* 


84 


76 


8 










■J I 








U665* 


52 


/ u 


-18 






R149* 


45 


A 
D 4 


-19 


From chart — select highest 








/ z 




temperature 




R120* 


50 


-22 


Transfer of momentum between 
objects 


13 


U769* 


79 


o o 


13 






R240* 


14 


A 


10 


Find density of wood block 












using ruler and balance 




U741* 


42 


34 


8 




No atypically 


low 


exercises 


17 


R319* 


65 


54 


11 


Speed of falling rock increases 




a843* 




15 


10 




R330* 


38 


29 


9 


Egg release is 14 days after 
menstruation 






33 




-23 






U853* 


33 


65 


-32 




Adult 


R418* 


62 


51 


11 


What constitutes chemical change 




U960 


38 


29 


9 




U919* 


70 


64 


6 






U956 


59 


56 


3 








56 




-14 






U929* 


35 


49 


-14 






R407* 


54 


68 


-14 


Testes produce sperm 




R411* 


48 


62 


-14 


Flower seeds develop from ovules 




U927* 


33 


50 


-17 



*Exercises also identified as atypical before balancing* 

Note: Each - - - line separates atypically high from atypically 
low exercises. 
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Exhibit 5-3 

Exercises Showing Atypical Effects, Balanced, 
for Education Level 2 

(At Least One Parent Attended but Failed 
to Complete High School) 



% Correct 



Exer- Level % Dif- 

Age cise # 2 Nat'l ference 



13 



17 



Content 



R13_7^ _ 13 12_ ^Mercury^is heavi.er than_water 

No atypically low exercises 

No_aty£i calmly high exercises 



58 



-13 



U723* 


45 


58 


-13 




R215* 


39 


52 


-13 


Flower seeds develop from ovules 


R220* 


26 


39 


-13 


Wood floats higher in salt water 










than fresh 


U742* 


15 


28 


-13 




U721* 


46 


60 


-14 




U760* 


28 


42 


-14 




R233* 


55 


70 


-15 


From chart — compare guinea 










pig weights 


U758* 


34 


49 


-15 


R327* 


37 


33 


4 


Molecules of air carry sound 


U839* 


24 


20 


4 


R351 


13 


27 


-14 


Kinetic-molecular theory 


U853* 


50 


65 


-15 


U828* 


23 


39 


-16 




R309* 


49 


65 


-16 


Who proposed natural selection^., 










in evolution 


U814* 


49 


65 


-16 




No atypically high 


exercises 


U958 


37 


50 


-13 




R440* 


38 


■^1 


-13 


Effect of reduced rabbit popu- 










lation on hawks and grass 


U956* 


42 


56 


-14 


R418* 


34 


51 


-17 


Speed of falling rock increases 


U960* 


11 


29 


-18 




U961* 


4 


23 


-19 





*Exercises also identified as atypical, before balancing. 

Note: Each - - - line separates atypically high from atypically 

low exercises* 
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atypical exercises (exhibit 5-4) shows that there are few 
exercises at any age that depart markedly from the median 
performance, and that those few exercises seem not to be closely 
related, one to the other, in content or form. 

Level 4; Least One Parent Educated Beyond High Schoo l 

For respondents at level 4, median balanced performance on 
science exercises is well above the nation at all four ages, 
about 4% above for ages 9, 13, and 17, and about 7% above for 
young adults. Many exercises are atypical in showing greater 
or lesser balanced effects as level 4; these are listed in 
exhibit 5-5 . 

Level 4 respondents performed especially well on exercises 
that demand a deep understanding of the scientific mex^od-- 
exercises concerning testing hypotheses (R147 and several 
unreleased exercises) , reading charts and graphs (R150, R233, 
and unreleased exercises) — and on a number of exercises that 
demand knowledge of particular facts and principles. Where 
respondents at level 1' show a deficit, respondents at level 4 
tend to perform unusually well. 

Level 0: Parental Education Unasc5i r tained 

The balanced median effects for parental education level 
0 are -4% at age 9, -5% at age 13, -9% at age 17, and -13% 
for adults. The^ deficit in performance, for respondents who fail 
to report the educational level of their parents becomes 
systematically larger with increasing age. It should be noted 
from exhibit 4-1 that only small numbers of 17-year-olds and 
adults fall in parental education level 0. 

Exhibit 5-6 lists the exercises that show atypical differ- 
ences from the median effect at each age. A number of exercises 
that show atypical deficits for level 0 seem to eiitail the 
ability to draw scientific inferences (e.g., R153, R131^R147, 
R235, R342, U853, U857) . Others call for specific knowledge 
about science (U633, U723, R309, R316) . These exercises have 
some resemblance to those that show atypical deficit at level 1. 
It seems likely tl-.at of the respondents who fail to report the 
level of their parents' education, many have parents with 
relatively limited numbers of years of schooling. 

Standardized Balanced Effects for Parental Education 

As in chapter 4, balanced effects may be expressed with 
reference to a standard population at each age level. Rather 
than comparing each group (at each age) with the national 
percent of success as a standard, we shall' use as a standard a 
population for which 25% of parents fall at each of the four 
educational levels. 
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The standardized medians for balanced results are shown 
in exhibit 5-7, The effects are somewhat reduced in size from 
those displayed in exhibit 4-10, as a consequence of balancing. 
As for the results of exhibit 4-10, however,, effects associated 
with the four levels of parental education appear to be almost 
identical, from one age to another. 
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Exhibit 5-4 

Exercises Showing Atypical Effects, Balanced, 
for Education Level 3 

{At Least One Parent Completed High School) 

% Correct 
Exer-- Level % Dif- 

cise # 3 Nat' 1 ference Content 
9 none 

13 No atypically high exercises 

U7Fl^ 35^ Tl ^ ""-6" 

Why milk is pasteurized 

Radio waves least upset plant 
and animal life 





R219 


34 


40 


-6 




U726* 


50 


56 


-6 




R213* 


51 


57 


-6 


17 


U814* 


71 


65 


6 










-b 




U856* 


53 


60 


-7 


Adult 


U963* 


32 


20 


12 




R405* 


81 


70 


11 




U9T0 


- To- 


- T8- - 





*Exercises also identified as atypical before balancing. 

Note: Each - ^ line separates atypically high from atypically 
low exercises. 
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Exhibit 5-5 



Exercises Showing Atypical Effects, Balanced, 
for Education Level 4 

(At Least One Parent Educated Beyond High School) 



% Correct 



Exer— 


Level 




% Dif- 




cise #■ 


4 


Nat' 1 


ference 


Content 


U633* 


75 


65 


10 




R159* 


33 


23 


10 


Writing a poem is not part of 










science 


R147* 


77 


68 


9 


To test an i<^ea, try it 


U626* 


80 


72 


8 


R150 


63 


55 


8 


From chart--sodium is least 










common j.n human body 


U634* 


71 


63 


8 




R152* 


61 


53 


8 


Because of vaccinations few 










people get smallpox 


U646 


46 


38 


8 




R141^ 


7 


7 


0 


Mixing water of 50° and 7 0° 










yields water of 60° 


U673* 


10 


11 


-1 




U653* 


20 


21 


-1 




U717* 


72 


63 


9 


- 


R213* 


66 


57 


9 


Radio waves least upset plant 










and animal life 


U754* 


72 


63 


9 




U773* 


45 . 


37 


8 




R233* 


78 


70 


8 


From chart — compare guinea pig 










weights 


U725* 


66 


58 


8 




R244* 


64 


56 


8 


What is a scientific theory 


U7 72^ 


92 


92 


0 




U853* 


77 


65 


12 




R342* 


75 ■ 


66 


9 


Effect of reduced rabbit popu- 










lation on hawks and grass 


U829* 


45 


36 


y 




R316* 


62 


53 


9 


Adrenalin is a stimulant to the 










heart 


R343 


65 


56 


9 


Differentiate fact from theory 
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Exhibit 5*5 (Continued) 



% Correct 



Lxer- 


Level 




% Dif- 




Age cise # 


4 


Nat ' 1 


f erencG 


Conti^nt 


R309* 


74 


65 


9 


Who proposed natural selection 










in evolution 


R319* 


50 


54 


-4 


Speed of fallTng rock increases 


Adult U958* 


75 


50 


25 




R419* 


71 


49 


22 


Hot water molecules move faster 


U927* 


69 


50 


19 




R426* 


55 


36 


19 


Molecular speied determineL 










different states of water 


U920* 


82 


64 


18 




■•s~^ R4 07* 


86 


68 


18 


Testes produce sperm 


'"^^■R4 23* 


58 


41 


17 


Classifying plar.ts and animals 










by structure 


U965* 


73 


56 


17 




R409* 


78 


62 


16 


Who proposed natural selection 










in evolution 


U965 


18 


20 


-2 




U919 


62 


64 


-2 




U921 


61 


63 


-2 





^Exercises • also identified as atypical before balancing. 

Note: Each - - - line separates atypically high from atypically 
low exercises. 
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Exhibit 5-6 

Exercises Showing Atypical Effects, Balanced, 
for Education Level 0 

(Parental Education Unascertained) 







% Correct 








Hj 2s. I. 


Level 








Arr<=» 


O 1 c: ^ ^ 
^ O ^ TT 


0 


Nat' 1 






9 


U637* 


58 


57 


1 






\J U ^ J 


22 


21 


1 

-L 






R141* 


8 


7 


1 


Mnxina w-^tf^T nf ^nr^ 70° 












yields water of 60° 




R15 3 


30 


38 


-8 


















R131* 


43 


51 


-8 


Cactuses lose little water 












throucrh 1 <^aves 




U63 3* 


57 


65 


-8 






R147* 


59 


68 


-9 


To 1"p'=5l" an idpa . tTV i t 


13 


R208* 


79 


74 


5 


lr\\J\^T^sr^ XClUll^l-i ^^»JOXUX%^ lll^^ll 




U723* 


42 


58 


-16 




R? "^5* 


45 


62 


-17 


Which weight experiment gives 












C!'4~V/^'nrTOC!"t~ ^c^Y\r^ ^ 

o L.X LJI 1^ o L. V X diii^ 


17 


U828* 


58 


39 


19 






U860* 


4V 


30 


17 






R325 


57 


41 


16 


Function of placenta in pregnant 












woman 




U805 


50 


B"2 " 


-32 






R3 09* 


31 


65 


-34 


Who proposed natural selection 












in evolution 




R316* 


28 


53 


-35 


Adrenalin is a stimulant to the 












heart 




R342* 


30 


66 


-36 


Effect of reduced rabbit popu- 












lation on hawks and grass 




U853* 


25 


65 


-40 






R344* 


15 


55 


-40 


Time a pendulum ' s swings 




U857* 


14 


55 


-41 



Adult R445* 26 11 
U961* 34 23 



15 
11 



Find density of wood block 
using ruler and balance 
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Exhibit 5-6 (Continued) 



I Correct 

Exer- Level % Oif- 

Age cis e 0 Nat' 1 f erence Content 

u957* 9 54 -45 
U956* 8 S5 -48 



Lxercises also identified as atypical before balancing. 

Note: Each - - - line separates atypically high from atypically 
low exercises. 
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Exhibit 5-7 



Standardized Balanced Median Effects for Parental 
Education at Four Ages 



4-1 

u 

0) 

m 
m 

w 

c 

as 

0) 0) 

n:l O 

Q) u 
u 

03 u 

-a 04 



N 
U 

03 
CO 



10. 



a 




o Age 9 
X Age 13 
# Age 17 
A Adult 



Level of Parental Education 
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CHAPTER 6 



SIZE AND TYPE OF COMMUNITY 



In previous reports respondents were divided into four sizes 
of community (SOC) categories: big city, urban fringe, medium 
city and smaller places. In this report a more detailed classi- 
fication* into seven categories is used. This classification 
of respondents is by school, not by pupil, and is based in part 
on information from the questionnaire filled out by the principals 
of participating schools (see appendix B of Report 1) . 

'0 

The first of these new categories is extreme inner city. A 
school was classified as an extreme inner city school if a high 
proportion of parents were either not regularly employed or were 
on welfare, and only a low proportion were professional or 
managerial. The majority, 85.6%, of the respondents from 
extreme inner city schools came from the former big city SOC 
category. 

The second new category is extreme affluent suburb. A school 
was placed in this category if a high proportion of parents 
were professional or managerial and only a few were factory or 
farm workers, or not regularly employed. More than half of the 
respondents from extreme affluent suburb schools came from the 
urban fringe (53.0%) SOC category, with big city (29.3%) and 
medium city (16.5%) making up the rest. 

The third new category, extreme rural, wa^s defined, by 
schools where a high p:;oportion of parents were farm workers. 
The percentage of respondents in this category coming from the 
SOC category of smaller places was 78.1%. 

These three extreme categories were defined so that approxi- 
mately 10% of the total sample fell into each. Thuj:, 70% of 
the respondents whose schools did not fall into one of thesa 
extreme categories were crouped by their original size of 
community classification. Thus, a' student in the big city 
classification was classified as rest of big city if he did not 
fall in one of these new extreme categories. Students in the 
urban fringe who did not fall into a new extreme category were 
classified as suburban fringe. Respondents from the smaller 
places category were classified a5j small cities if they did not 

*See- Appendix D for explicit definitions of these new classi- 
fications. 
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fall into one of the new extrSiiie categories. Since the medium 
city SOC category contributed few, if any, schools to the three 
new extreme categories, it was left virtually unchanged. 

Exhibit 6-1 displays the median effect (difference between 
the percent correct for the subgroup and the national percent 
correct) for each of the four original categories and for the 
new categories by each of the four age groups in the sample. 
It is clear from these data that the criteria for picking the 
three extreme types of communities did, indeed isolate groups 
of students v/hose perf orm.:3.nce v/as extreme. 

The concentration of the big city deficit in the extreme 
inner city is particularly striking. This large deficit and to 
a lesser extent that associa-ted with the extreme rural category 
clearly identifies those schools where American education 
faces its greatest challenge. Obviously many of the children 
in these schools are seriously deficient in their science 
achievement . 

The gap between the extreme affluent suburb and both the 
extreme inner city and the extreme rural is largest for 9-year- 
olds (7.2% vs. -15.1% and -6.3%). It decreases for the 13- and 
17-year-olds (6.3% vs. -13.7% and -6.1%) and (5.1% vs -7.4% and 
-3.5%), and then rises sharply for the adults (10.9% vs. -10.. 2% 
and -4.7%).. Since the adults were not attending school,, a 
different tWhnique was used to classify them into the extreme 
subcategorieVr-^This technique- together with the tendency for 
the better educated to escape from the extreme inner city and 
for the most successfixl to migrate to the extreme affluent 
suburb-probably underlies the large gap for the adult sample. 

Compared with adults, 17-year-olds show markedly smaller 
deficit in the extreme inner city and a. smaller advantage in the 
extreme affluent suburb. However, this may be due to the fact 
that there were substantially more very difficult exercises 
given to the 17-year-old sample than the other age groups (see 
page 10, chapter 1 of Report 1) . '^'^ery difficult multiple choice 
exercises tend to give the guesser an advantage over the respon- 
dent who tries to work it out. This hypothesis is supported by 
the fact that when we analyze the 69 exercises with national 
percents o£ at least 40% for 17-year-olds, the national effects 
-.are: for extreme inner city -10.4%, for extreme affluent suburb 
. +6.2%, and for extreme rural -6.1%. It may be that the 17-year- 
Qld respondents were not as representative of their age group 
as were those respondents at the other ages. 

The median effects for 9-year-olds for physical science 
exercises, biological science exercises, and the remaining 
unclassified exercises are shown in exhibit 6-2. This exhibit 
also displays the median ef f ec^ts for the four science obj ectives 
described in Report 1. The median performance of each of the 
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Exhibit 6-1 



Median Effects by Size and Type of 

for All Ages 



soc 


STOC 


9 


13 






17 




Adult 


Big city 




(-4.8) 


(-4. 


7) 


(■ 


-2. 


6) 


(-2.4) 




Extreme inner city 


-15.1 


-13. 


7 




-7. 


4 


-10'. 2 




Rest of big city 


-2.6 


-3. 


8 




0. 


3 


-2.9 


Urban" fringe 




{ 3.0) 


( 3. 


4) 


( 


2. 


4) 


( 3.2) 




Extreme affluent 
suburb 


7.2 


6. 


3 




5. 


1 


10.9 




"^Suburban fringe 


2.'' 


2. 


9 




1. 


0 


0.8 


Medium city 




( 0.8) 


( 1. 


1) 


( 


0. 


B) 


( 0.4) 




Medium city 


0.8 


1. 


9 




1, 


2 


0.4 


Smaller places 




(-1.2) 


(-1. 


1) 


( 


-2. 


1) 


(-2.8) 




Extreme rural . 


-6.3 


-6. 


1 




-3. 


5 


-4.7 




Small cities 


0.9 


0. 


5 




-1. 


4 


-2.7 



Note: Figures show difference between the groujp percent correct 
and the national percent correct. 
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Community 



Age 



Exhibit 6-2 



Median Effects for 9-Year-Olds by 
Type of Question and Objective 

Unclas- Objective* 
Physical Bj.ological sif ied 

science science exer >, i. ^ ^ £ 

Extireme 

Inner City -15.4 -15.7 -13.4 -15.7 -17.1 -11.5 -13.4 

E X 1 2r em e 

Rural -6.3 , -5.6 -8.2 -6.0 -7.2 -9.4 -6.2 

Rest of 

Big City -3.1 -2.6 -1.8 -3.1 -0.6 -3.0 0.8 

Medium 

Cities 0.4 1.4 1.0 0,9 O.S 2.3 0.2 

Small 

Cities 0.9 1.0 1,7 0.9 1.2 0,5 -0.1 

Suburban 

Fringe 3,5 2.0 1,7 3.0 2.4 1,3 1.1 

Extreme 
Affluent 

Suburb 7.3 7.0 9.7 7.2 7,2 8.5 2.6 



*1 - Facts and principles 

2 - Abilities and ckills 

3 - Understand investigative" nature 

4 - Atti :udes and appreciations 



f 
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seven community types is remarkably uniform across the different 
sorts of exercises. In particular, their relative performance 
on physical science exercises almost exactly matches their 
performance on biological science questions. The slight vari- 
ability observed in the rest of the table is quite likely due 
to the small number of exercises included in these groups. 

Exhibits 6-3, 6-4, and 6-5 show comparable data for 13-year- 
olds, 17-year-olds, and adults, respectively. The 13-year-olds, 
like the 9-year-olds, show virtually no change in pattern 
between physical and biological science or between objective 1, 
2, and 3. Objective 4 shows a much smaller range of variation 
but this is based upon only a few exercises. 

In the extreme inner city, 17-year-olds have a greater 
deficit in physical science than in biological science (-8.5% 
cornpared with -5.3%) while those in the extreme rural show just 
the opposite effects {-3.1% compared with -6.8%). Thus, the 
extreme inner city students do relatively better in biological 
science and the extreme rural ones better in physical science. 

Like the 9- and 13-year-olds, acquits show little difference 
in their relative performance on pnysical and biological science 
exercises except that the extreme inner city respondents showed 
a slightly smaller deficit in physical science. 

Exhibits 6-6 through 6-9 show the distribution of effects 
for all science exercises across all four ages for the three 
extreme categories and the medium city category. A comparison 
of the medians for all seven STOC categories was made earlier 
in this chapter, and the four distributions selected for exhibits 
6-6 through 6-9 were most interesting and indicative of the 
others , 
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Exhibit 6-3 



Median Effects for 13-Year-Olds by 
Type of Question and Ob j ective 

Objective* 

Physical Biological Other 

science science science i 2^ ^ 

Extreme 

Inner City -13.4 -14.9 -11.1 -14.1 -14.0 -14.6 -3.5 

Extreme 

Rural -6.5 -6.5 -4.1 -6.8 -6.1 -1.7 -2.4 

Kept of 

Big City -3.5 -5.4 -3.0 -4.1 -4.1 -3.0 -? . 2 

Medium 

Cities 1.6 3.4 .3 1.7 2.6 0.7 0.9 

Small 

Cities 0.7 -0.1 1.1 0.8 -0.6 2.2 -0.1 

Suburban 

Fringe 3.0 3.2 2.0 2.7 3.3 2.3 1.4 

Extreme 
Affluent 

Suburb 6.1 7.4 4.2 6.2 7.8 5.4 2.6 

*1 - Facts and principles 

2 - Abilities and skills 

3 - Understand investigative nature 

4 - Attitudes and appreciations" 
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Exhibit 6-4 



Median Effects for 17«Year-01ds .by 
Type of Questi6n . and-.Ob jective 



• Objective* 

Physical Biological Other 

science science science 1 2^ 3 



Extreme 

Inner City -8.5 

Extreme 

>Mral -3.1 

Big City -0.4 

^Ci^iies 1.2 

S:mali 

Cities . . -0.'8 

iJirbuxBan 

fringe 0.6 

^ilxtreme 
Affluent 

Suburb 5.2 



-5,3 
-6.8 
2.0 
1.3 
-2.3 
i.3 

4.8 



-6:6 -8.0 -7.9 -5.4 

-2.2 -3.3 -4.4 -9.2 

0.3 0,2 0.0 1.1 

-0.2 . 1.4 1.3 1.3 

-2.4 1.4 -1.6 -1.2 

3.5 O.B 2.3 -0.7 

5.1 4.9 6.1 7.0 



*1 - Faqts and principles 
2 - Abilities and skills , 

. 3 Understand investigative nature 
4 - Attitudes" and appreciations 



Exhibit 6-5 



Median Effects for Young Adults by 
Type of Ques uion and Objective 

Objective* 

Physical Biological Other 

science science science i. 2^ £ 

Extreme 

Inner City -10.0 -15.1 -8.2 -10.0 -13.5 -8.5 -7.1 

Extreme 

Rural -4.0 -5.1 -7.3 -9,9 -8,0 -7.6 -V>.9 

Rest of 

Big City -2.9 -3.1 -2,3 -3.5 -0.9 -3.6 -1.0 

Medium 

Cities 1.2 -1.0 0»2 -0*1 2.2 -0.2 0.9 

Small 

Cities -2.9 -2.2 -4.3 -2.9 -1.8 -7.1 -2.4 

Suburban 

Fringe 0.6 1.4 -2.0 0.3 4.6 -0.6 -1.6 

Extreme 
Affluent 

Suburb 9.5 10.0 14.1 11.0 6.6 13.0 9.9 

*1 - Facts and principles 

2 - Abilities and skills 

3 - Understand investigative nature 

4 - Attitudes and appreciations 
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Exhibit 6-6 




65 
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Exhibit 6-9 



Distributions of Extreme Affluent Suburb Effects 
for All Science Exercise'^ at Four Ages 
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13 



Age 



17 




No. of Exercises 
Median 



145 
7.2% 



122 
6.3% 



124 
5.1% 



119 
10.9 
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CHAPTER 7 



SIZE AND TYPE OF COMMUNITY GROUPS: BALANCED RESULTS 



The median effects associated with the seven types of 
community after balancing on region, sex, color, and parental 
educ^4ion are presented in exhibit 7-1. In general the balanced 
effects are about half of the unadjusted effects piesen.'ied in 
exhibit 6-1 of the preceding chapter. However, thore are 
several exceptions worth noting: the median balanz^ed effects 
for 13-year-olds, 17-year-olds and adultc- in the inner city 
are only about one third as large as the unadjusted medians, 
and that for extreme rural adultt; is essentially unchanged 
after balancing. 

The large reductior. in the type of community effe'^ts produced 
by balancing indicates that mu^h of the unadjusted effect was 
due to masquerading by other variables. The especially large 
reductions for inner city respondents suggest that the probl-^ms 
facing the inner city feghools come from the combination of 
several different sort's of factors, some associatea wi ch coloi , 
some with parental education, and some with region. The effects 
remaining after balancing are primarily associaited with features 
of the inner city community. Because a- vari jty of factors con- 
tribute to the inner city deficit, there may well be a variety 
of ways to improve the achievement of the children in these 
schools. , 

In a similar way, a number of factors actirg in combination 
are associated with the performance aeficit found for extreme 
rural communities and the performance advantage in the extreme 
affluent suburb. 

Exhibit 7-1 clearly shows the inner city and rural deficit 
and the affluent suburb advantage for t.ypical science exercises. 
However, not all science exercises show the same effects, and 
it may be instructive to consider atypical exercises, for v/hich 
the size and type of comm^jnity balanced effects seem large. 

Extreme Inner Cit y 

Exhibit 7-2 lists atypical exercises for the extreme inner 
city. At each age, those exercises for which inner city perfor- 
mance is atypically high are difficult exercises where the 
national percentage correct is no greater than 34% (with only" 
one exception, the apparatus exercise R341 at age 17)., The 
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Exhibit 7-1 



Balanced 
Type of 


Median Effects 
Community for 


by Size 
Each Age 


and 








o 


12 


Age 

17 




Adult 


Extreme Inner City 


-6.9 


-5.5 


-2. 


3 


-3 . 2 


Extreme Rural 


-3.4 


-2.8 


-1. 


6 


-4.5 


Rest of Big City 


-1.5 


-2.2 


0. 


4 


-1.9 


Medium Cities 


0.4 


0.9 


0. 


6 ' 


0.8 


Small Cities 


0.5 


0.7 


-0. 


2 


-2.0 


Subuirban Fringe 


0.7 


1.0 


-0. 


6 


-0.9 


Extreme Affluent Suburb 


4.4 


3.3 


.y 2. 


6 


5.6 
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Exhibit 7-2 



Exercises Shewing Atypical Effects, Balanced, 
for Extreme Inner City for All Ages 



% Correct 





Exer- 






% Dif- 






cise ■# 


ElCt 


Nat ' 1 


ference Content" 


9 


U655* 


23 


14 








U673* 


17 


11 


6 






R140* 


19 


14 


5 


Dead plants form coal 




R137* 
RlT9^ " 


37 
■ I6~ 


33 
'64 


A 

J- %j 


_ Mercury i^s^heavier than water 

TJ'vi^m o T" ^ e /^'^ ^ Vi n Vt r» 4- 




R104* 


73 


91 


-18 


Iron Qoesn't readily burn 




U644* 


23 


41 


-18 




R125* 


42 


61 


-19 


Plc'-ntsr aet watpr from soi 1 




U616* 


61 


80 


-19 




Ri46* 


55 


75^' 


-20 






U622* 


55 


75 


-20 






U665* 


49 


70 


-21 






U682* 


50 


72 


-22 






U630* 


45 


68 


-23 




13 


U.741* 


40 


,34 „ 


6 








32 




-17 






U760* 


• 25 


42 


-17 






R209* 


46 


65 


-19 


Sedimentary rock in layers 




R341 


85 


74 ' 


11 


Balance beam — weight in j^an? 












weight on hook 




U816^ 


4-5 


60 


-15 




'R314 


42 


56 


-16 


jBy natural selection, why 












giraffes have long necks 




R306* 


45 


66 


-23 


What doesn't increase supply of 
food 


Adult 


No a typically high 


exercises 




U961 . 


1 23 


-22 






U960 


6 


29 


-23 






U956* 


32 


56 


-24 






U957* 


26 


54 


-28 






R441* 


9 


48 


-39 


Time a pendulum's swings 




U955* 


28 


71 


-43 




*Exarcises also identified as 


atypical before balancing. 


t Extreme Inner City 




Note :' 


Each — ^ 


rine 


sersarates atypically high from atypi'cally low 


exercises . 












unusually good performance or these diffj.cult '-^xercises could be 
due to mor?i frequent guessing on the part of inner city 
respondents. | 

Many more exercises show atypically .low performance for^ I 
inner "city respondents,, especially for age 9 and for adults. | 
Included in this group at cjge C aire a number of 'exercises that' 
assess rather earthy aspects of science (R104, R125, R146, U644, 
U616. U622, U665, and U630} ; correct answers to many of these 
come not from the classroom, bv^t from everyday experiences of i 
children — experiences, however, that are less likely to be 
encounterrecl by i^^year-olds in an inner '''ity environment. rhe 
adult level, all 'j:cercises displayir.«:; '^rt atypical deficit are 
apparatus exaicises.. Perhaps, when appara^cus exercises are 
presented to adults during ah interview in their home , inner 
city adults tend to be soir.ewhat less, cooperative, and thus 
appear to perfor:n poorly on such exercises. 

Extreme Rural 

While relatively few exercises are ^atypic^i for rural respon 
dents (exhibit 7-3) , there is a tendency for those on which 
performance is atypically hign to tap knowledge that is., likely 
to be learned from general experience in rural communities. 
For example, the unreleased exercises on this list at, the adult 
level ask about: (1) the importance of green plants, (2) the 
effects of heating water in containers of varying shapes, and 
(3) the behavior 'of a wooden block floating oxx water • In 
contrast, an atypical rural deficit is seen for several exer- 
cises demanding graph reading, and for exercises R406 and R409, 
requiring considarahit^ mastery of vocabulary as well as detailed 
formal knov/ledge. 

Extreme Affluent Suburb 

In the affluent suburb, more exercises show atypically high 
than atypically low per f ormcxnce (exhibit ""'-8) . Eoth released and 
unreleased exercises that display unusua ',ly large a'if luent 
suburb advantages tend to assess relatively abstrac t facts and 
principles of science; the- affluent suburbs seem to perform 
best on exercise's that depend upoa science knowledge learned in 
the classroom. ^ » 

Other 3T0C Groups . ' . 

■ r ■ ; • 

For the rest of big city (exhibit 7-4), the med;Kum city 
(exhibit 7-5) , the small city (exhibit 7-6) , and the suburban 
\^ringe (exhibit 7-7) , relatively few exercises display atypical 
effects. No patterns of exercise content are easily dis.3rnible. 

\ " 



Exhibit 17- 3 



Exercises Showing Atypical Effects, Balanced, 
for Extreme Rural for All Ages 

% Correct . 





Exer- 






% Dif- 




Age 


cise # 


ERt 


Nat' 1 


ference 


Contend, 


S 


R157* 


51 


39 


- 12 ■ Recognize 


an untestable 




U666 


76 


66 


10 


















U662* 


^2 


81 


-19 




13 


U762* 


31 


22 


9 





No atypically low exercises — . 

17 No_aty£i calmly high exercises 

U853^ ?9 65 "16 ^ ^'^ ^ ~ 

U857* 39 55 -16 

Adult U922* 76 62 14 

R418 62 51 11 Speed of falling rock increases 

U928* 59 49 10 

U963 _ 30 20_' 1£ ■ ^ 

R406^ To 69 -19 Adrenalin Ta'a^^stimurant^to the 

heart 

R409* 34 62 -23 Who proposed natural selection 

in evolution 

^Exercise also identified as atypical before balancing. 

tExtreme Rural L_ 

Note:^ Each line separates atyp^.J-slly high from atypically 

low exercises. 
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.xhit)it 7-4 



Exercises Showing Atypical Effects, Balanced, 









for Rest 


of Bio City for All Ages 






% ( 


Correct 


- 




Exer- 






% Dif- 


Age 


cise # 


ICF 


r Nat'l 


ference Content 


9 


No atypically high 


exercises 




R157 


28 


39 


-11 Recognize an untestable statement 




U615 


68 


80 


-12 




U6 3 5 


49 


61 


-12 


13 


R234* 


77 


62 


15 Balance k)parQ-"-"VeiQht in oan* 










weight on hook 




U755* 


73 


59 


14 




U714 


79 


71 


8 




U7 51^ 


60 


73 


-13 




R215* 


38 


52 


-14 Flower seeds develop from ovules 


17 


R343* 


68 


56 


12 Differentiate fact from theory 




U865* 


30 


20 


10 




No atypically low 


exercises 


Adult 


U962* 


47 


22 


25 




U964 


33 


18 


15 




R449* 


19 


28 


-19 Often watch T.V. science programs 




U915* 


53 


74. 


-21 






/ 






*Exercises also 


identified as atypical before balancing. c 


tinner City 


Fringe 





Note: Each line separates atypically high from atypically 

low exercises- 
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Exhibit 7-5 



EKCircises Showing Atypical Effects-, BaMnced, 
for MecLxum City for Axl Ages 



% Correct 





Exer- 






% Dif- 


•e 


cise # 


MCt 


Nat' 1 


feiTPn CP Prtn 't"<=n t- 




R12 2* 


78 


71 


"7 Honey bees are helpful to man 




U64 0*- 


60 


53 


7 




No atypically low 


exercises 


13 


U744 


35 


28 


7 




U737^ 


36 


42 


"6 


17 ■ 


No atyDically high 


exerc isvrs 




R322* 


40 


48 


"8 Efficient use of food can cause 










overweight' " ^ ^ - 




R327 


25 




-8 Molecules of air carry sound 




U824 


36 


45 


-9 


Adult 


U960 


47 


29 


» 

18 ■ 




R441*- 


64 


48 


16 Timis a perfdulum's .swings 




U957 


67 


54 


13 V. - 




U956 


68 


56 


12 '■■ 



No atypica].ly low exercises 



*Exercise also identified as at^^ical before balancing, 
t Medium City . ' » 

Note: Each li.ne separates atypically high from atypically 

'low exercises. 
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Exhibit 7-6 



Exercises Showing Atypical Effects, Balanced, 
for Small City for All Ages 



% Correct 
Exer- % Dif- 

cise # set Nat'l ference Content 



U635 68 _61 7^ 

No atypicaTly low exercises 



13 No_aty£ica_ll;^ high_exercises 
U754^ 56 63 -7 

17 No__^"tyE^£^i-'-Z high_exercises 

R3 45^ 2 9 3¥ -ifJ Under stand ""conFervatron oflnatter 



Adult No_aty£ica]^lv high^exercises 
U957 38 '"54 -16 " 

U963 4 20 -16 



^Exercise- also identified as atypical before balancing. 
tSmall City 

Note; Each line separates atypically high from atypically 

low exercises. 
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Exhibit. 7-7 



Exercises Showing Atypical Effects , Balanced , 
for Suburban Fringe for All Ages 



%<rCorrect 
Exer- % Dif- 

Age cise # SFt Nat ' 1 f erence Content 

9 R123* 74 66 8 What scientists learn from 

^^E^i-'-E _ — _ 

R150* 46 55 -9 From chart-- sodium is least 

common in human body 

-'-'^ ^^^^^R^2P-1?-Y, il^2.^— — _ — — _ — _ _ 

U755 53 59 -6 

R237* 29 35 -6 Time a pendulum's swings 

U762* 14 22 -8 

17 No_aty£ical_lY^ hi2^h__exerci^ses__ 

R344* 41 55 -14 Time a pendulum's swrngs 

Adult R4 4_l* _ 62^ _48_ 1£ _ Time_a_pendul^um ' £ swi_ngs _ 

U921^ 48 63 -15 



*Exercise also identified as atypical before balancing. 
tSuburban Fringe 

Note: Each line separates atypically high from atypically 

low exercises. 
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Exhibit 7-8 



Exercises Showing Atypical Effects , Balanced , 
for Extreme Affluent Suburb for All Ages 



% Correct 



Exer- 






% Dif- 




cise # 


EASt 


Nat ' 1 


ference 


Content 


U645* 


56 


40 


16 




Ri2 3 


81 


66 


15 


What scientists learn from fossils 


R153* 


52 


38 


14 


Full sink wou]^(3 not explain 
faulty faucet 


R128* 


68 


55 


13 


Sun is a star 


U639* 


67 


54 


13 




R130* 


66 


53 


13 


Comfortable temperature: 7 0^ 


U648* 


50 


37 


13 




U651* 


29 


34 


-5 




R140* 


9 


14 


-5 


Dead plants form coal 



13 



17 



Adult 



U760* 


69 


42 


27 - 




U758* 


68 


49 






U755* 


51 


59 


-8 




R234* 


54 


62 


-8 


Balance beam--weight in pan; 










weight on hook 


U866* 


50 


30 


20 




R326* 


51 


35 


16 


Salt in water lowers freezing 










point 


U823* 


61 


46 


15 




U838* 


37 


22 


15 




U822* 


68 


54 


14 




R318* 


^15 


51 


14 


What constitutes chemical change 


U813* 


81 


67 


14 


R343 


45 


56 


-11 


Differentiate fact from theory 


U962* 


63 


22 


41 


t 


U956* 


90 


56 


34 




R441 


77 


48 


29 


Time a pendulum's swings 


U957* 


81 


54 


27 





No atypically low exercises 



*Exercise also identified as atypical before balancing. 
tExtreme Affluent Suburb 

Note: Each line separatees atypically high from atypically 

low exercises. 
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CHAPTER 8 



REGIONAL GROUPS: BALANCED RESULTS 



In Report 4, the patterns of correct response to the 
science exercises were examined for consistencies and d^crep- 
ancies among the nation's regions. In this chapter, we^ Examine 
the patterns in the regional samples after balancing for the 
disproportionate representation of four other character istics-- 
sex, size-and-type of community, education of parents, and 
color. To what extent can the regional effects noted in Report 4 
be considered to grow out of this lack of proportionality? 
What effects remain after appropriate balancing procedures 
(described in chapter 1 and in appendix B) have been applied to 
the regional data? 

The median offects at the four ages for the four regions, 
both the unadjusted and the balanced, are pictured in exhibit 
8-1. Thus, the effects of balancing have reduced the range of 
the regional effects by about one third. In general, the 
relative order of the medians within ages remains the same after 
balancing. The exceptions are the reversals of the order of the 
Northeast and the Central regions at age levels 9, 13, and adult; 
however, in none of these comparisons were the differences, 
either before or after balancing, large enough to be statis- 
tically reliable. The most striking difference between the 
unadjusted and the balanced effects is the sharp reduction in 
the Southeast deficit. Apparently, a substantial part of the 
deficit observed in the unadjusted data grows out of the fact 
that the^ effects of other characteristics are represented in 
different proportions in the different regional samples . On the 
other hand , a significant Southeast def icit (minimal for age 13 ) 
still remains after balancing. Also remaining is the relatively 
high median effect for adults in the West. 

The contrast between unadjusted and balanced effects 
presented in exhibit 8-1 involves all the science exercises. 
In Report 4 the unadjusted effects were also examined in terms 
of groups of exercises classified according to the four science 
objectives, and according to content, i.e., physical and bio- 
logical sciehc^e. The only instance in which the median effects 
for subgroups of exercises displayed a different picture from 
that pictured in the medians for all exercises was in the 
comparison between objective 1 and objective 2 exercises presented 
in exhibit 3-5 of Report 4. In 11 of the 16 comparisons, the 
median for objective 2 exercises was more extreme than the median 



Exhibit 8-1 

Median Effects for the Four Regions at Four 
Ages Before and After Balancing 

Before Balancing After Balancing 

Age 9 13 17 Adult 9 13 17 Adult 
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Exhibit 8-2 

Median Effects, After Balancing, for All Exercises, 
Objective 1 Exercises Only, and Objective 2 Exercises Only 

(Median Effects^ Before Balancing, for 
All Exercises, in Parentheses) 



Medians for 



All 



Group 


Age 


exercises 


UDJ • 


1 ie 
1*^ 




n ^ ^ 
^ X 7C 




Comparisont 


West 


Adult 


1.9 


(2.4) 


1. 


4 


2. 


0 


0. 


6 


bigger 




Northeast 


n 


1 . 5 


(2.6) 


1 . 


2. 


1 . 


9 


0 . 


7 


bigger 




Central 


13 


1.2 


(1.9) 


1. 


2 


1. 


3 


0. 


1 


bigger 




Central 


9 


1.0 


(1.6) 


0. 


1 


1 . 


5 


0. 


8 


bigger 




Northeast 


9 


0.8 


(2.3) 


0. 


5 


1. 


9 


1. 


4 


bigger 




Northeast 


13 


0.8 


(2.0) 


0. 


4 


2. 


1 


1. 


7 


bigger 




West 


17 


0.7 


\ 

(1.7) 


-0. 


1 


1. 


3 


1. 


4 


across 


zerott 


West 


9 


0.6 


(d.o) 


1. 


0 


-0. 


6 


1. 


6 


across 


zerott 


Northeast 


;^.duit 


0.3 


(0.6) 


0. 


6 


-0. 


5 


1. 


1 


across 


zerott 


Ce^itral 


Adult 


0.3 


(0.7) 


0. 


1 


1. 


6 


1. 


7 


bigger 




Central 


17 


0.1 


(0.0) 


0. 


2 


-0. 


1 


0. 


3 


across 


zerott 


West 


13 


-0.8 


(-0.5) 


-0. 


7 


-0. 


8 


0. 


]. 


bigger 




Southeast 


13 


-1.8 


(-4.7) 


-1. 


2 


-4. 


2 


,3. 


0 


bigger 




Southeast 


Adult 


-2 . 4 


(-4.9) 


-2. 


2 


-2. 


5 


JO- 


3 


bigger 




Southeast 


17 


-2.5 


(-4.9) 


-2. 


2 


-3 . 


1 


0. 


9 


bigger 




Southeast 


9 


-3.1 


(-5.1) 


-3 . 


3 


-3 . 


2 


0. 


1 


sinallertt 



*i 

^Objective 1 - Facts and principles 
**Dbjective 2 - Abilities and skills 

tObjective 2 compared to objective 1 

ttExceptions to the rule 
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for objective 1 exercises. This contrast remains after balancing, 
as illustrated in exhibit 8-2. However, the absolute differences 
tend to be smaller, as would be expected as a result of the 
general reduction in the size of the effects. (The median of 
the absolute differences is 1*35% for the unadjusted median 
effects and 0.85% for the balanced median effects.) 

The median effects presented in exhibits 8-1 and 8-2 provide 
one way of viewing the results of balancing. As was true of our 
study of the unadjusted data, it is also instructive to examine 
not only the medians but also the distributions from which the 
medians are derived, to identify exercises falling so far toward 
the extremes of the distributions as to be considered at;) ical , 
and to ask whether or not it is possible to discover any >tematic 
distinctions between exercises atypical above and those atypically 
below the median effects. 

In the sections which follow, we report the results of these 
analyses, taking the regions in order Northeast, Southeast, 
Central, and West and then comparing the balanced data reported 
here with the unadjusted data in Report 4. 

Northeast 

The distributions of balanced Northeast regional effects 
are presented in exhibit 8-3. The exercises falling beyond the 
boundaries marked by the arrows on exhibit 8-3 and so defined as 
atypical, are identified in exhibit 8-^4, together with percent 
correct for national, adjusted percent correct for the Northeast, 
and the percent difference. 

In general, the picture presented by the balanced data for 
the Northeast is £;imilar to that provided by the unadjusted 
data of Report 4. At ages 9, 1*3, and 17 the medians are about 
1% above the national medians (0.8%, 1.5%; in contrast to 
median effects of about 2% for the unadjusted medians. These 
are small but statistically reliable effects. The median for 
adults in the Northeast differ from adults in the country as a 
whole. The same conclusion was reached on the basis of the 
median of the unadjusted effects {0.7%). 

Of the 18 exercises identified as atypical, all but four 
v/ere also identified as atypical in the distributions of unad- 
justed effects, and two of these were near the line marking 
.atypical items in the unadjusted distributions. Six of the 
eight exercises classified un-.^er objective 2 show an advantage 
for the Norcheast. In contrast, only four of the nine exercises 
classified under objective 1 show an advantage. in general, 
then, the summary statement of Report 4 applies also to atypical 
exercises after adjustment: "there appears to be a tendency 
for the Northeast to show an atypical advantage on a higher 
proportion of objective 2 than of objective 1 exercises." 
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Exhibit 8-4 



Exercises Showing Atypical Effects, Balanced, 
for the Northeast for All Ages 



% Correct 



Exer- 


North- 




% Dif- 




cise # 


east 


Nat' 


1 f erence 


Content 


R128* 


64 


55 


9 


Sun is a star 


R153* 


46 


38 


8 


Full sink would not explain 
faulty faucet 


U669* 


59 


52 


7 




U667* 


70 


63 


7 




U626* 


79 


72 


7 




U64 6 


45 


38 


7 




U676 


47 


41 


6 










-5 


Houseflies spread serious 
diseases 


R135* 


33 


38 


-5 


Effect of submerged rock on 
water level 


U635* 


52 


61 


-9 





13 


U760* 


54 


42 


12 




U758* 


61 


49 


12 














R222* 


30 


38 


-8 


17 


U861* 


38 


29 


9 










-6 


Adult 


R420* 


54 


44 


10 




R441* ~ 


"31 ~ 


~48 


-17 ~ 



Matter behaves as" Tf""inade 

of atoms 
By natural selection, why 

giraffes have long necks 



Reason why car engine stops 
Function of placenta in 
Time a pendulum's ^wTngs 



*Exercise also identified as atypical before balancing. 

Note: Each line separates atypically high from atypically 

low exercises* 
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Southeast 

Distributions of balanced effects for the Southeast region 
are presented in exhibit 8-5. Exercises showing atypical 
balanced effects for che Southeast are listed in exhibit 8-6\ 
As in the case of the Northeast samples^ the balanced data for 
the Southeast tend to show a similar if somewhat less extreme- 
picture than the unadjusted data found in Report 4. Whereas 
the unadjusted median deficits were about 5%^ the balanced 
median deficits are about 2.5%, The median balanced effects^ 
therefore, are only half as large as the unadjusted median 
effects, but they are still large enough to indicate that 
performance in the Southeast tends to be below the national 
performance on science exercises. 

Altogether , there are 26 science exercises identified as 
atypical after balancing, a number somewhat smaller than the 42 
reported before balancing. Of the 26 exercises, 17 were also 
identified as atypical before balancing and nine were not. As in 
the case of the data for the ':<Iortheast , most of the fluctuations 
can be attributed to the influence of random sampling. There 
is, however, one difference between the lists of atypical 
exercises in the two reports worth noting. In Report 4, we 
noted a difference in the rela^tive frequency of exercises 
classified under objective 2 exhibiting atypical advantage or 
deficit in the Southeast. 

All nine of the objective 2 exercises identified in Report 4 
(before balancing) as atypical showed a deficit. In contrast, 
of the six objective 2 exercises, showing atypical effects for 
the Southeast after balancing, three show an advantage and three 
a deficit. Thus, the general tendency for objective 2 exercises 
to show a greater deficit than objective 1 exercises in the 
Southeast, as reported in exhibit 8-2., cannot be attributed to 
the influence of a few atypical exercises; the effect must be a 
general one involving the objective 2 exercises as a group. 

Central 

Distributions of balanced effectij for the Central region 
are shown in exhibit 8-7, and those exercises identified as 
atypical after balancing are listed in exhibit 8-8. The medians 
at ages 9 and 13 are less extreme than before balancing (1.0% 
and 1.2% in contrast to 1.6% and 1.9%), but it is still possible 
to say, on the basis of the balanced data, that the Central 
region performs about 1% better than the nation as a whole at 
ages 9 and 13^ At aqes 17 and adult, the changes as a result of 
balancing are small, a reasonable result in view of the fact 
that the effects before balancing were small. 

Of the 11 exercises identified as atypical and listed in 
exhibit 8^8, five show a Central deficit and six a Central 
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Exhibit 8-6 



Exercis.es Shov/ing Atypical Effects, Balanced, 
for the Southeast for All Ages 



% Correct 



Exer- 


South- 




% Dif- 




cise # 


east 


Nat ' 1 


ference 


Content 


R140* 


24 


14 


10 


Dead plants form coal 


U677* 


40 


33 


7 ■ 




U653* 


26 


21 


5 




U638* 


61 


56 


5 




R138* 


39 


35 


4 


Clearing skies follow cold 
front 


R154* 


20 


16 


4 


VJater freezing point same as 


R152 


57 






ice melting point 


53 


4 


Because of vaccinations few 
people get smallpox 


U635 


51 


61 


-10 




R156* 


69 


79 


-1 0 


Recognize a scientific 


R129 








explanation 


44 


54 


•-10 


Interpreting a thermometer 
reading 


R12 8 


45 


55 


-10 


Sun is a star 


U619 


65 


77 


-12 




U626* 


58 


72 


-14 




R123* 


50 


66 


-16 


What scientists learn from 



fossils 



R217* 


55 


48 


7 


Burning gasoline in a car 










creates heat 


R227* 


31 


25 


6 


Center of memory is cerebrum 


U718 


67 


61 


6 


U738 


45 


40 


5 






53 


62 


-9 


Balance beam — weTght in pan; 










weight on hook 


U736 


33 


44 


-9 




U816* 


67 


60 


7 




U840* 


26 


20 


6 




U861* 


18 


29 


-11 





Adult ^io_st^y£ica_llv high exercises 

U960 15 29 ^14 

U920* 50 64 -14 

U924* 39 56 -17 
*Exercise also identified as atypical before balancing. 

Note: Each line separates atypically high from atypically 

low exercises* 
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Exhibit 8-8 



Exercises Showing Atypical Effects, Balanced, 
for the Central for All Ages 



> % Correct 
Exer- % Dif- 

l Age cise tt Central Nat' 1 f erence Content 

9 



U635 


69 


61 


8 




R123* 


73 


66 


7 


What scientists learn from 
fossils 


R129 


61 


54 


7- 


Interpreting a thermometer 
reading 


U<n40 


48 


53 


-5 




R126 . 


54 


. 59 


-5 


Center of. Earth is hot 


R138* 


30 


35 


-5 


Clearing skies follow cold 



front 



U758* 44 49 --5 

U760* 36 42 -6 

17 U8^8^ 46_ 3^9'^ Z _ . 

No atypically low exercises 

Adult U960* 44 29 15 

U962*_ 32_ 22 10 _ _ 

- -No atypically low exercises 



^Exercise also identified as atypical before balancing. 

Note: Each line separates atypically high from atypically- 

low exercises. 
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advantage. Eight were among the 12 exercises identified as 
atypical in the distributions of unadjusted effects in Report 4, 
and three appear as atypical here for the first time. Two of the 
four atypical exercises classified as objective 2 show a positive 
effect and two a negative. Here, even more than in the case of 
thB effects before balancing, the general impression is one of 
lack of pattern. Thus, we can only note that there are some 
large deviations from the median without propo5i:ing hypotheses 
to explain the deviations. 

West 

The distributions of balanced effects for the West are 
pictured in exhibit 8-9 and the atypical exercises are listed in 
exhibit 8-10. The balancing has reduced somewhat the median 
effects at ages 17 and adult (from 1.7% and 2.6% before balancing 
to 0.7% and 1.9% after balancing), and when these percentages 
are compared with the estimates of fluctuations to be expected 
from sample to sample, it is no longer possibly to say that the 
performance in the West is any better than that for the country 
as a whole. The medialis at ages 9 and 13 were not reliably 
different from the national performance before balancing, and 
the same remark applies to the medians after balancing. 

Of the 13 exercises identified as atypical after balancing, 
il were so identified before balancing* When the four exercises 
showing an atypical advantage for the West are compared with 
the nine showing an atypical disadvantage, we identified no 
meaningful contrasts in terms of content of exercises. 
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Exhibit 8-10 

Exercises Showing Atypical Effects, Balanced, 
for the West for All Ages 



% Correct 



Age 


Exer- 
cise # 


West 


Nat'l 


% Dif- 
ference 


Content 


9 


U640* 


63 


53 


10 






U647* 


43 


36 


7 






U667^ 


55 


63 


-8 






U669* 


44 


52 


-8 





13 No_atypica_lly hi2.h_exerc_ises 



U718 


54 


61 


-7 


U711* 


68 


76 


-8 


U728* 


44 


53 


-9 


Lit54* 


72 


64 


8 


US 14* 


72 . 


65 


7 


US 15 2"^ 


19 


25 


-6 


U82 8* 


3 2 


39 


-7 



Adult No c) typ^ical^ly^ — 

r44"5^ 1 11 -10 Find density of wood block 

using ruler and balance 

U960 17 29 -12 



*Exercise also identified as atypical before balancing. 

Note: Each line separates atypically high from atypical?.y 

low exercises. 
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CHAPTER 9 



SEX DIFFERENCES AFTER BALANCING 



Since the sexes are relatively well balanced across the 
other classifications^ we do not anticipate 'and co not f^nd any 
remarkable effects due to balancing . Exhib5 t 9-1 shows the 
medians for each age . Consequently , a .discussion of the 
balanced sex differences must lead to the same general conclu- 
sions as those reached from discussion of the unadjusted sex 
differences, as given in. Report 4. Accordingly, no further 
comments are necessary here. 




Exhibit 9-1 

Median Sex Differences for Four Ages, 
Unadjusted and Balanced 

Median difference: Male minus female 



Age Unadjusted Balanced Change 

9 1.8 1.4 0.4 

13 2.9 2.4 0.5 

17 5.0 4.0 1.0 

Adult 9.9 9.2 0.7 
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This report presents unadjusted " a.nd balanced results at 
four ages (9, 13, 17 , and adult) , classified in several ways. 
The unadjusted results are classified by color (Black and 
non-Black), parents' education (five levels), and size-and- type 
of community (seven levels) . The balanced results are classified 
by sex (male and female) , and geographic region (four areas) , as 
well as the three unadjusted classifications mentioned above.* 
Our concern is to describe how well these groups of respondents 
perform on science exercises. 

In the interest of clarity and ease, we wan+: to make as 
few niamerical comparisons as will adequately describe what we 
are to observe. When there are only two groups in a classi- 
fication, as with sex divided into male and female, one numerical 
comparison will suffice. We need only compare one group with 
the other for each exercise. For sex we will use. the difference: 



Sex 
Difference 



% Success I 
For Males ! 



% Succe 
For Fema 




For example, one exercise given at age 9 shows 76% of the males 
and 68% of the females responding succes'^f ully . Using the 
procedure above: 



Sex 
Difference 



76% 



681; 



8% 



Where there are' more than tvro groups in a classification, 
such differences are not as simple to use. With four groups, 
such as four regions or four community sizes, simplicity is 
gained by comparing the percentage of success of z-espondents in 
each group with that in the nation as a whole. We shall call 
the number thus obtained an effect. One example would be 



Northeast 
Effect 



Northeast 
% Success 



National 
% Success 



*See Report 4 for unadjusted results for male-female and 
geographic region . 
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For one exercise which 66% of 9-year-olds in the Northeast 
and 55% of all 9-year-olds answered successfully, the regional 
effect is 

Northeast 

Effect . = 66% - 55% = 11% 

What Should Be Described? 

For people of the same age , when we compare the performance 
of a group--classif ied by geographic region or size of community — 
with all people of that age in the country, the results are 
different for each exer.cise. They are different, but not too 
different. Even though some exercises show an advantage for the 
group, others a deficit, there is often a clear tendency for the 
exercises that wjei^^dre considering as a whole to lean one way or 
the other. We describe this tendency by reporting the typical 
behavior of exercises. 

Since not all exercises show the same advantage or deficit 
for the group, description of the typical behavior of all 
exercises in the class is usually not enough . We can learn 
something more from studying individual exercises for which the 
group advantage or deficit differs substantially from this 
typical behavior. If we use an app7;opriate rule to identify 
these a typical exercises , it will be worthwhile to mention each 
of them individually, telling something of their nature and 
reporting what advantage or deficit each shows when the group is 
compared with the whole country. This concentration on atypical 
exercises is particularly helpful because exercises showing close 
to typical behavior are often lacking in distinctive flavor. 
Atypical exercises, by contrast, often tell us something. 

When we deal with the two sexes, we do the same things, 
except we shall compare males with females, rather than a group 
with the whole country, both ^.n terms of typical behavior and in 
terms of atypical exercises. 

Classes Of Exercises 

Another approach to the discussion of our results is 
possible because the science exercises may be separated into 
classes by some common feature. We may investigate the behavior 
of the exercises for each class, and then ask hdw'this behavior 
changes from one class to another. 

Only a few separations into classes are considered in this 
report . The classification of items into physical and biological 
exercises is a separation by content. Another classification is 
provided by the four science objectives : * 

*Norris, Eleanor L. (Ed.) Science Ob j ectives . Ann Arbor, 
Michigan: Committee on Assessing the Progress of Education, 
1969. 
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1* Know fundair.ental facts and principles of science; 

2. Possess the abilities and skills needed to engage in 
the processes of science ; 

3. Understand the investigative nature of science; 

4 . Have attitudes about and appreciations of scientists , 
science, and the consequences of science that stem 
from adequate understandings. 

Although other ways of classifying the exercises might be 
at least as useful in describing sex differences, regional 
effects, or community size effects, we have not found them. 



What Shall Be Typical? 

In the interest of simplicity and clarity, we choose to 
summarize the behavior of a set of exercises by the m edian (or 
midpoint) of the differences from national results. The median 
is a summary value such that an , equal number of the values 
summarized lie on either side of it. For the 9-year-old sample-- 
for instancjln^^^ in exhibit 2-1, the^te^ were 145 exercises--72 
exercises where "the Black effect was greater than -14.5% and 72 
less than -14.5%. Because the same numlJer of exercises had 
Black effects greater than -14.5% as well as less than -14.5% we 
call -14.5% the median Black effect and use it to summarize all 
145 effects for the Black group. 

The median is an effective typical value because: (1) it 
is easy to understand and (2) it is little affected by the 
presence or absence of unusual values. 



What Shall Be Atypical? 

Only a sample of each age group responded to each exercise, 
yet National Assessment results are concerned with whole popu- 
lations-- some tifues with all 9 -year-olds in the country , some- 
times with all 13"-year-old boys, sometimes with all 17-year- 
olds in the Northeast, sometimes with all adults in small cities, 
and so on. Thus there must always be a step from sample to 
population in interpreting National Assessment results, allowing 
for the inescapable small differences between sample results and 
population facts. This inevitably raises statistical consid- 
erations--considerations which ought to guide our approaches, our 
exposition, and our statements without interfering excessively 
with the reporting and discussion of results. 



The samples taken in National Assessment were well designed 
scientific probability samples. As a result, it would be 
possible to assess how large the differences between sample 
value and population value are likely to have been for almost any 
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quantity vie chose t^o calculate from the results — for example, 
a Black effect. Results of such calculations have guid(i;d the 
choices in this report of how large a departure from typical 
fitiedian) behavior qualifies an exercise for separate considerati- 
Appendix C discusses consi-derations involved in, and the exact 
nature of, this choice. 

The chosen rule sets cutoffs determining which comparisons 
we call atypical . The cutoffs found by applying the chosen rule 
are shown by two arrows on each picture of the distribution of 
comparisons. Exercises corresponding to comparisons outside 
these cutoffs are discussed individually. 




APPENDIX B 



BALANCING 



Purpose 

This is the first of the National Assessment reports to 
look at more than one characteristic of respondents at a time. 
We try to reduce the extent to which effects of one character- 
istic can masquerade as effects of another characteristic. 

Proportionate Numbers Create No Problems 

\We first look at some simple examples of what can--and, 
appropriate circumstances, must — happen. Assume that we are 
assessing the effects of one characteristic that classifies 
respondents into either group A, B, or C and of another char-- 
acteristic that classifies respondents into either group U or V. 
The two characteristics combined to form six combination sub- 
groups: AU, AV, BU, BV, CU, and CV. For our first set of 
examples, we assume that the numbers of respondents in each of 
these six subgroups are as shown in the upper left corner of 
exhibit B-1. (The first three examples all use this same 
distribution of supposed observations.) Note that, for every 
group on the first characteristic (A, B, and C), representation 
from group U stands in a fixed proportion to that from group V. 
Also note that we have proportionate numbers from groups A, 
B, and C in both groups U and V. 

In example 1, every U subgroup — whether it is AU, BU, or 
CU — has 6 0% success, and every V subgroup — v/hether it is AV, 
BV,. CV — has 40% success. Since the AU subgroup is of size 100 
and. has 60% success, it includes 60 successes (60% of 100) . The 
numbers of successes for the other five subgroups are calculated 
similarly. To find the total number of successes for all of 
group U, we have only to add up the number of successes for each 
of the three subgroups in U, finding 60 + 120 + 180 = 360. 
Similarly, we add to find the total number of respondents in 
group U, 100 + 200 + 300 = 600. The unadjusted percentage of 
success in group U is then 360/600 = 60%. When we do the same 
for group V and for groups A, B, C (defined by the other 
characteristic), we find very satisfactory results. Group U 
has a percentage of success that is 12% better than the overall 
percentage of success, 48%. Group V has a percentage of success 
that is 8% poorer than the overall percentage of success. No 
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Exhibit B-1 



Some Examples of Simple Behavior When 
the Numbers of Cases are Proportional 



Example 1 



# of cases 



% of 
success 



# of 
successes 



U V 








u 


V 






A 100 150 


250 


60% 40% 


A 


60 


60 


120 


48% 


B 200 300 


500 


60% 40% 


B 


120 


120 


240 


48% 


C 300 450 


750 


60% 40% 


C 


180 


180 


360 


48% 


600 900 


1500 






360 


360 


720 












60% 


40% 




48% 


Example 2 




















% of 




# 


of 






# of cases 


success 




successes 






U V 








U 


V 






A 100 150 


250 


40% 40% 


A 


40 


60 


100 


40% 


B 200 300 


500 


50% 50% 


B 


IQO 


150 


250 


50% 


C 300 450 


750 


60% 60%. 


C 


180 


270 


450 


60% 


600 900 


1500 






320 


480 


800 












53 . 3% 


53.3% 




53 . 3% 


Example 3 




















% of 




# 


of 






# of casec 


success 




successes 




U V 




\ 




U 


V 






A 100 150 


250 


50% 30% 


A 


50 


45 


95 


38% 


B 200 300 


500 


60% 40% 


B 


120 


120 


240 


48% 


C 3C0 450 


750 


70% 50% 


C 


210 


225 


435 


58% 


600 900 


1500 






380 


390 


770 












63. 3% 


43.3% 




51.3% 
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difference appears among groups A, B, and C. Thus, in this 
case, the percentages in the five marginal totals that correspond 
to simple unadjusted results are completely consistent with the 
percentages for the six subgroups that generate the data. 

Example 2 shows how differences between A, B, and C will 
have the proper size when computed as above with proportionate 
numbers in the subgroups. In this example, the difference 
between U and V is zero. 

Example 3 is a little more complex. It assumes that each 
U subgroup does 2 0% better than the corresponding V subgroup, 
but that there are also differences when we change from C to 
B or B to A. Again directly computing results from the groups 
(A, B, C, U, V) causes no trouble. The overall success is 
51.3%. The percent of success for group A is 38%, that is, 
13.3% below the total percent of success. The percent of 
success for group U is 63.3, that is, 12% above tJ^e total percent: 
of success. And when we add together thci three effects, we 
obtain the percent of success for the AU subgroup, i.e., 51.3% - 
13.3% -f 12.0% = 50%. The result of combining the effects for any 
of the other combinations is evjually consistent. This consistency 
is something special; t depends essentially on the sizes of the 
six subgroups being *;propor tional . 

Disproportionate Numbers Can Make Trouble 

We now turn to a different set of examples, where the 
numbers of respondents are not proportional within the categories. 
The upper left-hand corner of exhibit B"'2 shows the new pattern . 

Example 4 shows the same pattern for percent of successes 
as example 1 where there is no difference between A, B, and C 
within levels U and V and equal differences between U and V 
at each level of A, B, and C. Thus, in this example the marginal 
totals accurately reflect the difference between the U and V 
subgroups but they give quite an erroneous picture of the A, 
B, and C effects. By examining the distribution of the number 
of cases it is evident that the marginal (unadjusted) ratio of 
successes to cases for category C is in actuality measuring the 
effect of V. Thus, in this example, one variable masquerading 
for another has affected the unadjusted estimates of the effects 
of the A-B-C categories. This would not be apparent if we had 
not simultaneously looked at both the A-B-C and U-V classi- 
fications . Given disproportionate numbers , the true difference 
between U and V both appears as itself and masquerades as dif- 
ferences among A, 3, and C. 

Example 5 shows how differences among A, B, and' C can 
masquerade as differences between U and V. Example 6 shows 
how each of two sets of real differences can both appear for 
themselves and masquerade as the other. 
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Exhibit B~2 



Some Examples o/f^^'^Simple Behavior 
When the Numbers are Disproportionate 



Example 4 

# of cases 



% of 
success 



# of 
successes 



U V 








U 


V 






A 100 100 


200 


60% 40% 


A 


60 


40 


100 


50% 


B 50 150 


200 


60% 40% 


B 


30 


60 


90 


45% 


C 0 200 


200 


60% 40% 


C 




80 


80 


40% 


150 450 


600 






90 


180 


270 












6 0% 


40% 




45% 


Example 5 




}f 






















of 






# of ca 


ses 


suSiSiess 

\ 




sv.c 


cesses 






U V 




\ 




u 


V 






A 100 100 


200 


40% 40% 


A 


40 


40 


80 


40% 


B 50 150 


200 


50% 50% 


B 


25 


75 


100 


50% 


C 0 200 


200 


60% 60% 


C 




120 


120 


60% 


150 450 


600 






65 


235 


300 












43.3% 


52.2% 




50% 


Example 6 




















% of 




# 


of 






# of cases 


success 




successes 






U V 








U 


V 






A 100 100 


200 


50% 30% 


A 


50 


30 


80 


40% 


B 50 150 


200 


60% 40% 


B 


30 


60 


90 


45% 


C 0 200 


200 


70% 50% 


C 




100 


100 


50% 


150 450 


600 






80 


190 


27 0 












53 .3% 


42.2% 




45% 
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In none of these examples do we obtain the percent of 
success for the subgroups by combining the overall effect and 
the unadjusted group effects as we did when the numbers of cases 
were proportional. Thus, in example 6, the overall percent of 
success is 45, the A effect is 5% lower and the U effect is 
8.3% higher , and when we add we get 45% - 5% + 8.3% = 48.3%, not 
50% as we would like. However, if we assume an A effect of 
^10%, a B effect of 0%, a C effect of +10%, a U effect of +15% 
and a V effect of -5%, then the combinations are consistent. For 
example, 45% - 10% + 15% = 50% for the AB combination and 
45% + 10% - 5% = 50% for the CV combination. It has been possible, 
then, to find a set of effects that has removed the distortion 
due to disproportionate numbers in the subgroups , at least in 
this instance where the group effects are the same from one 
subgroup to another. 

The Need 

Disproportionate numbers in real population groups produce 
the problem that we have been discussing. For example, a 
larger fraction of Blacks are found both in the inner city and in 
rural areas. Larger fractions of Blacks also have low parental 
education. When we look at the whole group of Black children, 
some of the deficit shown by the group comes from effects 
characterizing inner cities or rural areas and some from the 
effects of lower parental education. 

We should do what we can to see through this sort of 
confusion. It was possible to adjust the data in example 6 
simply by inspecting the patterns of percentages for the six 
subgroups. These patterns, however, were based on hypothetical 
data. In real data there are often complex effects for combina- 
tions in the subgroups over and above the group effects. Further-- 
more, the hypothetical data of example 6 involved only two 
characteristics. In real data there are often more than two 
characteristics im^olved, and the simple inspection technique 
that we used in finding the relationships in example 6 will no 
longer work. There are, however, a number of ways to carry out 
computations that can help to reduce the distortion due to 
disproportionality . We next state the intended result of the 
calculation, then we say how we go about obtaining it. 

A Balanced Fit 

We intend to find group effects (expressed in percentages) 
that, when combined by addition with each other and with the 
overall percentage of success, give fitted percentages of 
success that correspond with the actual data in one simple way: 

if we choose any group by a single characteristic, 
say group A in example 6, and if we use the fitted 
percentages and the actual numbers of cases to 
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calculate the number of successes for each subgroup 
that involves group A, and if we then add these 
calculated numbers of successes, the total number 
of successes over all subgroups will be the same 
as the total actually observed in the data . 

Suppose , for example , that we have 60 0 cases distributed 
as in the upper left of exhibit B-3 and that the total number 
of successes (270) is distributed among groups A, B, C, U, and 
V as shown in the upper right of exhibit B~3, The overall 
percentage of success is 270/600 or 45%. Now, if we let A, B, 
C, U, and V stand for the five group effects (in percentages) , 
our definition of balancing says that the fitted number of 
successes in the AU subgroup may be represented by 100 (451 -f 
A -f U) , the fitted number of successes in the AV subgroup is 
100 (45% -f A -f V) , and the number of fitted successes in both 
subgroups by the sum of these two. Moreover, the definition 
says that this sum must be equal to the number of successes 
actually observed, which is 80. Thus, we can write the equation 
100 (45% -f A -f U) + 100 (45% + A + V) = 80. We can also write 
four other equations, two for the rows involving groups B and C 
and two for the columns involving groups A and U, as illustrated 
in the middle section of exhibit B-3. If we try to solve these 
five equations for the five effects, however, we find that we 
are not able to obtain a unique solution. The equations are not 
independent since the three equations for the rows must sum to 
the overall total of 27 0 and likewise the tv/o equations for the 
columns. We have only three independent equations, two for the 
rows and one for the columns, and we need two additional inde- 
pendent equations in order to find a single solution. 

There is an additional requirement established by our 
definition. Overall, the effects of A, B, and C must balance 
each other since all that is left is the overall effect; like^ 
wise, the effects of U and V must balance each other. This 
means that we can write two additional equations based on the 
numbers of cases in the margins of our table . These are : 

150 (45% + U) -f 450 (45% -f V) = 270 

and 

200 (45% -f A) + 200 (45% -f B) + 200 (45% -f C) = 270 
and these reduce to 

150U + 450V = 0 

and 

200A -f 200B -f 200C = 0 

If we then use these two last equations along with any two of 
the three equations based on the rows and any one of the two 
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Exhibit B-3 



An Example of the Procedure Used in 
Obtaining a Balanced Fit 

Example 7 

# of 

# of cases successes 





u 


V 






u • 


V 




A 


100 


100 


200 


A 






80 


B 


50 


150 


200 


B 






90 


C 


0 


200 


200 


C 






100 




150 


450 


600 




80 


190 


270 



The representation of the fitted successes in the subcrroup 



Subgroup Representation 

AU 100 (45% + A + U) 

AV 100 (45% + A + V) 

BU 50 (45% + B + U) 

BV 150 (45% + B + V) 

CU 0 

CV 200 (45% + C + V) 
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Equations 
Combination 

AU + AV 100 (45% + A + U) + 100 (45% + A + V) =80 

BU + BV 50 (45% + B + U) + 150 (45% + B + V} =90 

CU + CV 200 (45% + C + V) = 100 

AU + BU + CU 100 (45% + A + U) + 50(45% + B + U) + 0 = 80 

AV + BV + CV 10G(45%+A+V) + 150(45%+B+V) + 200(45%+C+V) = 190 

A + B + C 200 (A + B 4- C) =0 

U + V 150U + 450V = 0 

The solution 

A effect = -10% 
B effect = 0% 
C effect = +10% 
U effect = +15% 
V effect = - 5% 
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equations based on the columns, we have five independent 
equations and five unknowns that can be solved for the five 
unique balanced effects. The solution gives the values in the 
lower part of exhibit B-3, and it is observed that the solution 
is the same as we obtained in example 6. 

It is easy to look at this example and say, "Ah, I see. 
This example is the same as example 6, and the balanced fit 
reproduces the number of successes in each subgroup exactly." 
Notice, however, that in example 7 there are no numbers of 
successes given for the subgroups. It is true that the balanced 
fit of example 7 does produce an exactly matching fit to the 
actual data of example 6, but it is also a balanced fit to data 
that have different frequencies of success in the subgrryups . 
Exhibit B-4 shows several patterns of successes all having the 
same balanced fit. 

Balancing does not remove all the complications from the 
data. It does strip off the concealment involved in masquerades 
caused by disproportionate numbers. 

More General Cases 

We have considered only two characteristics, one with two 
groups, the other with three. The same technique extends to 
more groups* Extensions to more characteristics involve more 
complicated equations, but the interpretations are equally 
straightforward, so long as we realize that what we balance are 
the n\imber--or equivalently the percents — of success for all 
groups defined by any single characteristic (and not necessarily 
for any subgroup defined by two or more characteristics) . 

If, as in this report, we balance 

sex, two groups 

color , three groups (including unascertained) 
region , four groups 

education, five groups (including unascertained) 
size and type of community, seven groups 

we have taken a long step to reduce masquerading. But it would 
be wrong to think that we have gone the whole way, for: 

we have not yet used all, merely most, of the 
characteristics that were collected ; 

we have only used these characteristics in a rather 
limited way (we could have used more than four 
regions, more types of communities, final classifi- 
cations of years of school completed, for instance) ; 

we know that the overt characteristics are less 
than perfect measures of the variables that should 
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Exhibit B-4 

Examples of Possible Observations 
Leading to fhe Same Balanced Fits 



The pattern of number of cases (for all examples) 



U V 



A 


100 


100 


200 


B 


50 


150 


200 


C 


0 


200 


200 




150 


450 


500 



Some alternative patterns of nu rrtb e r s of successes 
Example 8* Example 9 





U 


V 




A 


50 


30 


. 80 


B 


30 


60 


90 


C 




100 


100 



u 

A 40 
B 40 
C — 



V 

40 
50 
100 



80 
90 
100 



80 190 270 
Example 10 
U V 



80 190 270 

Example 11 
U V 



A 


70 


10 


80 


A 


80 


0 


80 


B 


10 


80 


90 


B 




90 


90 


C 




100 


100 


C 




100 


100 




OK 


190 


270 




80 


190 


270 



Remarks 



Since the numbers of successes for the A, C,,U, V single- 
characteristic groups are 80, 90, 100, 80, 190 for each of the 
four examples, the fit of exhibit B-3 is a balanced fit for each 
pattern. 



*Example 8 is the same as example 6. 



107 



concern us (years of school is a less-than* perfect 
measure of parental education, which is itself a 
less than perfect measure of either home attitude 
toward education or available aid in homework) ; 

there are other characteristics of importance, e.g., 
economic status, that were not measured. 

The deficiencies of balancing are clear; it cannot be the 
final answer. But the step from unadjusted comparisons to 
balanced ones is a long step from outward appearance toward-- 
toward, not to — inward realities. The problem facing the schools 
is usually better shown by unadjusted values; the effectiveness 
of the educational process for a group is of ten better indicated 
by balanced results. 

Another Way to Look at Things 

We can look at the examples we have just considered in 
another, more detailed, way. Instead of comparing U with V 
overall, we can compare them for A, B, and for C separately. In 
example 6 of exhibit P. -2, for example, AUs do 20% better than 
AVs, BUS do 20% better than BVs , and CUs do 20% better than CVs . 
Unless we allow disproportionate numbers to distort the picture, 
we ought clearly to say that, unless A, B, or C is considered to 
be relevant to the comparison of U with V, Us do 20% -better 
than Vs. If the balanced effects did not say this, we would 
dislike them. Since they do say Us do 20% better than Vs, we 
are satisfied — at least so far as this example goes. 

Some of the alternative examples of exhibit B-4 are 
something else again. Example 9, for instance, has percent 
success values as follows: 

U V 

, A 40% 40% 

B 80% 33.3% 

C ? 50% 

Here AUs are equally successful with AVs, while BUS are 46,6% 
more successful than BVs. Clearly the overall answer for Us 
compared to Vs has to be between 0% and 46.6%. Surely 20%, as 
recorded by the balanced fit, is a reasonable possibility. 

When we consider the other two examples of exhibit B-4, 
we find 20% between 60% and -33% for example 10, and between 
80% and ? for example 11. When the comparison of Us with Vs 
is not the same for AUs and AVs a^ for BUS and BVs or for 
CUs and CVs, any simple statement about the comparison of Us 
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and Vs has to be a compromi a. The statement provided by a 
balanced fit is simple, convenient, and useful. It does not — 
and cannot--tell the whole story. 

If the comparisons of Us with Vs is the same within A 
as within B as within C, then, when the number of cases in 
all 2 X 3 = 6 cells are proportional , the unadjusted (marginal) 
comparison of Us ;^ith Vs will be the same as: 

the comparison within A. 
the comparison within B . 
the comparison within C. 
the balanced-fit comparison . 

This is fine for any way or ways in which the numbers are 
proportionate • 

Thus it is natural to describe balanced effects as what 
would occur "if the other characteristics were represented 
proportionately in each group." This statement is only exact 
in the important special case we have just discussed, but it 
gives a useful impression in most cases. 

In example 9 of exhibit B-4, the comparison of AUs with 
AVs (40% =^ 40%) is different from that of BUs with BVs (80% 
7^ 33.3%). Here, different ways of changing the number of cases 
in each cell to make the number proportionate will give dif- 
ferent overall comparisons (different marginal comparisons) . 
Exhibit B-5 shows three results that could arise by throwing 
cases out of cells until the numbers become proportionate , if 
successes and failures have an equal chance of being thrown out. 

We see from the examples (examples 12 , 13 and 14 ) that just 
how we make the numbers proportional does affect the overall 
comparison of Us with Vs, but really not by as much as we might 
fear. There is no single comparison that "would hold if numbers 
were made proportionate," but even so, the picture of what 
would happen *'if things were made proportionate" is a good 
mental picture to have about the meaning of balanced effects. 
So long as we keep it a little vague, it serves us quite well 
indeed. 

A more complete description cannot stop with effects for 
single groups. In a later report we will give an account of 
how some of the larger subgroups, subgroups formed of respon- 
dents belonging to t.wo specified groups , such as "graduated 
from high school" and "West" — perform, telling whether the 
overall effects for the two groups tell the story well enough 
when their values are added together, or whether we need to 
add in interactions as a third term. 
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Exhibit B-5 



Two Results Obtainable by Discarding Cases 
From Examp.i.6 9 in an Unbiased Way 



Example 12 
# of cases 



150 150 

Example 13: . 
# of cases 



% overall 



53.3% 37.8% 
Dirf erence 14 . 5% 



% overall 
success 



# of 
successes 





U 


V 




U 


V 


A 


100 


100 


A 


40 


40 


B 


50 


50 


B 


40 


17- 


C 


0 


0 


C 


0 


0 



80 57- 



# of 
successes 



u 


V 






U 


V 


A 50 


100 




A 


20 


40 




B 50 


100 




B 


40 


33 + 




C 0 


0 




C 


0 


0 


















100 


200 


60% 3 6.7% 




60 


73 + 








Diff^^hce 23.7% 








Example 


14 














% overall 




# 


of 


# of 


cascis success 




successes 


U 


V 






U 


V 


A 70 


100 




A 


28 


40 




B 49 


70 




B 


39- 


23 + 




C 0 


0 




C 


0 


0 


















119 


170 


56.1% 37.1% 




67- 


63 + 








Difference 19% 




i 
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APPENDIX C 



CHOOSING THE CUTOFFS 



Our observed percentages of success are based on samples, 
but our interest is in whole populations. Thus our information 
is always less than complete. As a result, conclusions stated 
either in numbers or in words are approximate rather than exact, 
slightly blurred rather than sharp. Whatever rule we choose to 
fix the cutoffs, and thus to decide which comparisons are 
atypical and worth serious consideration, the performance of that 
rule will be less than perfect. 

However we choose a cutoff rule, there will be some chance 
that an exercise whose population value is typical or close to 
typical will have a sample value classed as atypical by our 
rule. The more one tries to avoid this by stiffening the rule, 
thus calling fewer exercises atypical, the more often sample 
values will fail to be recognized as atypical when their popu- 
lation values are far from typical. This is a general problem 
that arises whenever sample results (including differences and 
effects) are examined in detail. 

In choosing a rule, we must take account of these two kinds 
of unfortunate consequences. We may also need to consider what 
sizes of differences or effects are likely to be practically 
meaningful. Besides discussing individual exercises, we want to 
discuss the groups of exercises found atypical in a given 
direction. Our choice of cutoffs affects our ability to describe 
such groups. The rule whose details are given below was chosen 
with some attention to all these considerations. 

Variances and Standard Errors 

The samples drawn for the first assessment, both in and out 
of school, involved careful designs in which the country was 
divided into areas called strata. For each exercise, results 
were obtained for two groups of respondents in each stratum.* 

*The two groups come from two "primary sampling units" in each 
stratum. (See appendix *C of Report 1.) Because of the existence 
of planned half strata and the loss of some primary sampling 
units, losses that were duly allowed for in the estimation 
procedure, it was necessary to "collapse" a few pairs of strata 
by combining two original strata in one stratum for the purpose 
of assessing sampling variability. 



To assess sampling variability, it is necessary to estimate 
a suitable measure of how far the results of the 1969-70 
administration of a science exercise would have varied had 
different groups of respondents been selected in any or all 
strata. Our concern has to be with other possible selections 
according to the same random procedures used in the actual 
administration. 

In this situation, the use of two groups of respondents per 
stratum allows us to calculate an estimate of such a measure of 
variability, the sampling variance, defined as the average 
square of the difference betv/een a single administration and the 
average result for all possible administrations . 

In this report, our immediate concern is with the estimated 
sampling variance of sex differences, regional effects', size 
and type of community effects, color effects, and parental 
education effects . It is equally useful to look at the standard 
errors of these comparisons,, defined as the square roots of the 
estimated sampling variances. 

These estimated sampling variances or standard errors will 
vary from one exercise to another for several reasons, including: 

1. Different sample sizes for some exercises. (The 
time-consuming apparatus exercises were given 

to fewer respondents. Certain other exercises 
were excluded in a few jurisdictions . Shortness 
of class periods occasionally caused omission 
of later exercises. ) 

2. The automatic decrease in variance accompanying 
very easy or very hard exercises. 

3 . Larger or smaller differences in the sizes of 
actual school -to- school differences . (In some 
cases these presumably reflect the greater or 
lesser effect or extent of curriculum variations.) 

4. Sampling variations inevitable in the process 
of estimating sampling var iabil ity . 

For reference, appendix E presents estimated standard errors 
for effects, both unadjusted and adjusted, for each science 
exercise classified by objective. 

Kinds of Cutoffs 

If we wanted to look at results only for individual exer- 
cises, it would be natural to compare each deviation (whether 
from zero or from a typical comparison) with a suitable multiple 
of the standard error available for that deviation. Doing this 
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v;Quld correspond, in terms of cutoffs, to using a separate 
cutoff for each exercise. This would have been quite possible 
by using the estimated standard errors given in appendix E. 
However, it was felt that this would involve more details and 
complications than would be warranted by the 4;ikely gains. 
(The interested reader can use appendix E as a basis for his ov7n 
experiments with this type of analysis.) 

Instead, the choice was to use a single pair of cutoffs for 
each class of exercises considered in the report. To do this, 
it was necessary to combine standard errors or estimated sampling 
variances across exercises. For this purpose, it is appropriate 
to use a combined standard error, which is the square root of 
a combined estimated sampling variance, this latter being the 
arithmetic mean of the estimated sampling variances for the 
exercises involved. 

Size of Cutoffs 

The easiest way to use the combined standard error to fix 
' ±h'e cutoffs is to start fron the corresponding median and lay 
off a suitable multiple of the combined standard error on either 
side. If the multiple is large, very few exercises will be ^ 
falsely called atypical, but we will fail to take advantage of 
the information provided by intermediate deviations. 

There are circumstances under which we might be able to 
make good use of quite small multipliers, even fractions. When 
comparing one subclass of exercises well below the median with 
another subclass of exercises well above the median, larger 
subclasses could be helpful, so helpful as to outweigh the fact 
that a substantial fraction of each subclass would fall in that 
subclass purely by accident. 

A multiplier of about two is conventional for many purposes. 
This choice leads to about 5% of all items entering one dis- 
tinctive subclass or th-^ other purely by chance. Since we deal 
with classes of about 120 exercises (and since 5% of 120 is 6) 
this would mean an average of about three "atypically high" 
exercises and about three "atypically low" exercises purely by 
chance. Thus this choice seemed more likely to confuse the 
picture than to clarify it. 

Another practical detail should concern us . Our exhibits 
show only the integer percentage for a difference or effect. 
Thus all comparisons between 3.0 and 3.9 are shown as 3, for 
example , if our cutoffs are to be simply related to our pictures , 
they should fall at the end of such a range . (A cutoff between 
2.9 and 3.0 or one between 3.9 and 4.0 is easrly pictured and 
used. One between 3.4 and 3.5, or at 3.74, is not.) 
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As a result, it was decided to set the cutoffs in the 
following v^ay : 

1. start from the median comparison for all the 
exercises involved, 

2. move up and down distances equal to 2.5 times 
the combined standard error , and 

3. move further in each direction until the cutoff 
divides one integer-percent range from the next. 

Thus if the median were -1.7% and the combined standard 
error was 1.9^^, we would find first 

(2.5) X (1.9%) = 4.75% 

then 

-1.7% - 4.75% -6.45% and -1.7% + 4.75% = 3.05% 
and then place the arrows between the intervals 

-7.9 to -7.0 and -6.9 to -6.0 
on the low side and between the intervals 

3.0 to 3.9 and 4,0 to 4.9 

on the high side (j..e., locate the whole percentage interval 
containing the computed cutoff point, then place the cutoff 
arrows at the point dividing that interval from the adjusted 
interval furthest from the median) . 

For each clasi;. of more than 100 exercises, an average 
of perhaps one or t.vo exercises can be expected to fall outside 
these cutoffs because of sampling fluctuations. (The smaller 
sample sizes for apparatus exercises can increase these numbers 
somewhat.) A substantial number of exercises that we might 
have liked to consider atypical will of course fail to be 
recognized as such . Some such compromise is , however , necessary . 
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APPENDIX D 



DEFINITION OF TERMS 



The major reporting categories used in this report are: 
age, region, size and type of community (STOC) , sex, color, and 
parents ' education . 

Age 

Four age levels were assessed. Three of these were in-school 
--9-, 13-, and 17-year-olds — and two — 17-year-olds and young 
adults — were out-of-school , sometimes referred to as the house- 
hold sample. The criteria used or the operational definitions 
of the four ages are as follows: 

9-year-olds — Born between 1/1/60 and 12/31/60 
13-year-olds — Born between 1/1/56 and 12/31/56 
17-year-olds — In-school: Born between 10/1/51 and 9/30/52 
0ut-of-3chool : Born between 10/1/50 and 
9/'^0/51 and not enrolled in school in March '68 
OR born between 10/1/51 and 9/30/52 and not 
enrolled in school in March 1969. 
young adults — Born between 7/1/33 and 6/30/43 



Region 



Four 


geographical regions 


are 


being used for all 


age levels 


Northeast, 


Southeast, Central , 


and 


West. The states 


falling in 


each of these four regions, for 


reporting purposes , are as 


follows: 










Nor theast 


Southeast 




Central 


West 


Del. 


Ala. 




N.D. 


Hawaii 


Maine 


Ark. 




S.D, 


Alaska 


N.H. 


Fla. 




Iowa 


Idaho 


Vt. 


Ga. 




Kansas 


Mont. 


D..C. 


Ky. 




Minn . 


Wyo. 


Md. 


La . 




Mo. 


Utah 


N.J. 


Miss . 




Neb. 


N.M. 


N.Y. 


N.C. 
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Nev. 


Conn . 


S.C. 




Ind. 


Cal. 


Pa. 


Tenn. 




Mich • 


Ore. 
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Northeast 



Southeast 



Central 



West 



Mass • 




Ohio 
Wis . 



Wash. 
Ariz . 
Colo. 
Tex. 
Okla. 



This classification is that used by the Office of Business 
Economics, Department of Commerce; the names for regions used by 
the OBE differ from National Assessment names for three regions: 



Size and Type of Community (ST OC) 

Previous National Assessment reports have given results for 
four sizes of community (SOC) : big cities, fringes around big 
cities, medium size cities, and smaller places. These are defined 
in National Assessment Reports 4, 5, and 6. This (Report 7) 
and subsequent reports integrate three "extreme types of com- 
munity," each composed of approximately 10% of the population, 
with four sizes of community representing the remaining 70% of 
the population into a seven-category STOC classification. 

NOTE: The four "size of community" categories within the 
STOC classification are not equivalent to the four "size of 
community" categories within the SOC classification used in 
previous reports since the latter did not have thet three "extreme 
types of community" extracted from them. 

The definitions of the extreme types of community were 
derived from an occupation question for both the in-school and 
out-of-school samples (see exhibit D-1) . 

By classifying schools rather than counties, it was possible 
to identify much more uniform groups. One extreme group was 
selected in each of the three directions indicated by the 
exploratory analysis: (1) schools where high proportions were 
farmers or farm workers; (2) city schools where a high propor- 
tion of parents were either not regularly employed or on welfare, 
and only a low proportion were professional or managerial; 
(3) near-^city and city schools where a high proportion of parents 
were professional or managerial and only low proportions were 
factory or farm workers, not regularly employed or on relief. 



National Assessment 



OBE 



Northeast 
Southeast 
Central 
West 



Nor 
Sou 
Gre 
Wes 



thern Atlantic 
theast 

at Lakes and Plains 
t and Northwest 
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Exhibit D-1 



Correspondence Between In-- and Out-of-School 
Occupation Categories 



Principal ' s questionnaire 
categories Code 

Professional or managerial A 
personnel 



Sales, clerical, technical, B 
or skilled workers 



Out-of-school and adult 
categories (from PS Manual) 

Professional , technical , and 

kindred Managers, officials 
proprietors (except farm) 

Clerical and kindred 
Sales workers 
Craftsmen , foremen , and 
kindred 



Factory or other blue collar C 
workers 



Operative and kindred 
Service workers, private 

household, and other 
Other laborers 



Farm workers 



Farmers and farm managers 
Farm laborers and foremen 



Not regularly employed 
On welfare 



E Unemployed 

F Receiving welfare 



00 



Unclassified 



Smaller extreme groups would have been more extreme; larger 
extreme groups would have had better determined percentages of 
success. The sizes of the three extreme groups, close to 10% 
of all those assessed, were chosen as a compromise between more 
extremeness and better determination. 

STOC 1. Extreme Rural; 9.6% of all respondents, of vjhich: 
7.5% come from SOC category small places , which contained 28.3% 
of all respondents ; 1.7% come from SOC category medium size 
city , which contained 25.7% of all respondents; 0.4% come from 
SOC category fringes around big cities , which contained 22.8% 
of all respondents . This STOC category represents respondents 
attending schools or living in a community having a po pulation 
less than 3500 . They are among those ranked highest on the 
rural index D - (C + 2A) . These communities lie within counties 
comprising three of the four old SOC categories as follows: 

Fringes around big cities; 0.4%-~withih SMSA* 
counties containing a city with population greater 
than 200,000 but outside the city limits. 

Medium size city : 1.7%--within all other SMSA 
counties not containing a city with population 
greater than 200, 000 and other non-SiMSA counties 
containing a city with population between 25,000 
and 50,000. 

Small places ; 7.5% — within all other non-SMSA 
counties not included in medium size city. 

i 

STOC 2. Extreme Inner City: 9.9% of all respondents, of 
which: 8.5% come from SOC category big city , which contained 
23.2% of all respondents; 0.9% come from SOC category medium 
size city , which contained 25.7% of all respondents; 0.5% come 
from SOC ^category fringes around big cities , which contained 
22.8% of all respondents. This STOC category represents 
respondents attending schools or living in a community within the 
city limits or residential area served by a city with a population 



*SMSA — Standard Metropolitan Statistical Areas. An economic amd 
social unit which is metropolitan in character and contains at 
least: (a) one central city with 50,000 inhabitants or more, or 
(b) two cities having contiguous boundaries with a combined 
population of at least 50,000. The smaller city must have a 
population of at least 15,000. The SMSA includes the county 
in which the central city is located, and adjacent counties 
that are found to be metropolitan in character and economically 
and socially integrated with the county of the central city. 
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greater than 150^000 , They are among those ranked highest on 
the inner city index E + F ~ A. These communities lie within 
counties comprising three of the four old SOC categories as 
follows : 

Big city: 8.5% — within the city limits of a 
city with population greater than 2 00^000. 

Fringes around big cities: 0 . 5% — within SMSA 
counties ccntaining a city with population 
greater than 200,000 but outside the city limits. 

Medium size city: 0.9% — within other SMSA 
counties no^ included above. 

STOC 3. Extreme Affluent Suburb: 10.0% of all respondents, 
of which: 5.3% come from fringes around big cities which 
contained, 22.8% of all respondents; 2.9% come from big city , 
which contained, 23..?% of all respondents; 1.6% come from 
medium size city , which contained 25.7% of all respondents. 
This STOC category represents respondents attending schools or 
living in a community within the city limits or residential area 
served by a city with a population greater than 150,000 . They 
are among those ranked highest on the suburb index A - (C + D 
+ E + P) . These comJTiunities lie within counties comprising three 
of the four old SOC categories as follows: 

Big city; 2.9% — within the city limits of a 
city with population greater than 200,000. 

Iringes around big cities: 5.3% — within SMSA 
counties containing a city with population 
greater than 200,000 but outside the city limits. 

Medium size city: 1.6%--within other SMSA 
counties not included above. 

STOC 4. Rest of Big City: 11.8% of all respondents. This 
is the residue of the' big city SOC category, which contained 
23.2% {8.5% went to extreme inner city, and 2.9% to extreme 
affluent suburbs) . This STOC category represents other respon- 
dents attending schools or living in a community within the city 
limits of a city greater than 200,000 not categorized above by 
STOC 2 or 3 as extreme inner city or extreme affluent suburb 
respectively. 

STOC 5. Suburban Fringe: 16.6% of all respondents. This 
is the residue of the f ringes~around big cities SOC category, ^ 
which included 2 2.8% of all respondents (5.3% went to extreme 
affluent suburbs, 0.5% to extreme inner city and 0.4% to extreme 
rural) . This STOC category represents all other respondents 
{not categorized above by STOC 1, 2 , or 3 as extreme rural , 
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extreme inner city, or extreme affluent suburb) attending 
schools or living in a community within an SMSA county containing 
a city with a population greater than 200,000 , but outside the 
city limits. This category includes some communities between 
3,500 and 150,000 and some non-farm communities less than 3,500. 

STQC 6. Medium City: 21.4% of all respondents. This is 
the residue of the medium size city SOC category, which included 
25. 7 o of all respondents . (1.7% went to extreme rural, 1.6% 
to extreme affluent suburb, 0.9% to extreme inner city.) This 
STOC category represents all other respondents (not categorized 
above by STOC 1, 2, 3, or 5 as extreme rural, extreme inner city, 
extreme affluent suburb, or suburban fringe) attending schools 
or living in a community served by a city with a population 
between 150,000 and 200,000 , and within either an SMSA county 
having a city v/ith a population less than 150,000 or a non-SMSA 
county having a city with a population of at least 25,000. This 
category includes some communities between 3., 500 and 150,000 
and some non-farm communities less than 3,500. 

STQC 7. Small City: 20.7% of all respondents. This is the 
residue of the small places SOC category, which contained 28.3% 
of all respondents (7.5% went to extreme rural) . This STOC 
category represents all other respondents (not categorized above 
by STOC 1, 2, 3, 5 , or 6 as extreme rural, extreme inner city, 
extreme affluent suburb, suburban fringe, or medium city) 
attending schools or living in a community within a non-SMSA 
county. This category includes communities between 3,500 and 
25,000 and some non-farm communities less than 3,500. 



\ Data are reported for Blacks and non-Blacks. Color classi- 
fication was based on observation of the respondent at the time 
of the administration by the field staff member who administered 
the package . 

Parents ' Education 

All data are reported in terms of the highest level of 
education of either parent or the guardian of a 17-year-old 
respondent. Parents ' Education is reported as follows : 



Sex 



Data are reported for Males and Females. 



Color 



Level 1 : 
Level 2 : 



Level 3: 
Level 4 : 



Less than or equal to 8th grade 

More than 8th grade but not a high 

school graduate 

Graduated from high school 

Some formal education beyond high school 
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APPENDIX E 



NATIONAL PERCENTAGES OF SUCCESS, EFFECTS, AND STANDARD 
ERRORS FOR REGION, SEX, SIZE AND TYPE OF COMMUNITY 
(STOC) , COLOR, AND PARENTS' EDUCATION — UNADJUSTED 
AND BALANCED — FOR RELEASED AND UNRELEASED 
EXERCISES BY OBJECTIVE 



Data on the percentage of success , ef f e^:ts , and standard 
errors of effects (unadjusted and balanced) for each science 
exercise are shown for the four ages--9, 13, 17, and adult. 
Within each age group they are identified as released or unre- 
1 eased exercises by a three-digit number preceded by an R 
(released) or a U (unreleased) as follov/s: 



Age 


Released 


Unreleased 


9 


RlOOs 


U600S 


13 


R200S 


U700S 


17 


R300S 


U800S 


Adult 


R400S 


U900S 



(All released exercises are reproduced in NAEP Report 1.) 

The exercises are listed according to objective 1 first, 
followed in turn by exercises for objectives 2, 3 and 4, For 
each objective, the released (unreleased) exercise numbers are 
assigned in order from highest to lowest perce.ntage of national 
success. Thus, within each age the exercises are divided into 
two main groups--released and unreleased — with the exercises 
listed in order of decreasing national success within each 
objective. 

In this appendix , the first line corresponding to each 
exercise shows the identifying number, national percentage of 
success, and for released exercises, a short description of the 
exercise content . The next four lines show the unadjusted 
effects and standard errors and the balanced effects and standard 
errors , respectively , for the five classifications of respondents 
included in this report (region, sex, STOC, color, and parental 
education). Exercises which overlap other exercises afc^ other 
ages are listed in appendix C of Report 4. Identif icaMon of 
physical and biological science classifications are also 
included in appendix C of^ Report 4. 
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